Qual vs Quant
Using Better Data to Build Better Products
Laura Klein
laura@usersknow.com
@lauraklein
Who’sTired Of Hearing About
Data?*
*be honest
A/BTesting
A/BTesting
Funnel
Analysis
A/BTesting
Cohorts
Funnel
Analysis
A/BTesting
Cohorts
Funnel
Analysis
Blah
A/BTesting
Cohorts
Funnel
Analysis
Blah
Blah
A/BTesting
Cohorts
Funnel
Analysis
Blah
Blah
Blah
Sorry. I’m GoingToTalk About
Data Some More.
Better Data
==
Better Products
Well…Sometimes
What Data Can’t Do
Replace Design
Replace Listening to Users
Tell Us What We Should Be Building
What Data Can Do
Inform Design
Inform Research
Tell Us What We Should Be Investigating
How About An Example?
A Funnel
A Checkout Funnel
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
A Leaky Checkout Funnel
100%
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
-20%
-10%
-50%
-33%
-25%
== ???
WhatThat Means toYou
100 Buyers
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers
== 18 Buyers
100To 18????
How Do We FixThis?
Identifying a Friction Point
100 Buyers
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers
== 18 Buyers
Stop It
Quantitative Data Qualitative Data
Understanding the Friction Point
Quantitative Data
WHAT
Qualitative Data
WHY
How Can You Use
Them Together?
Step 1: Identify the Biggest Problem
Step 1: Identify the Biggest Problem
Use Funnel Metrics
The Biggest Problem
100 Buyers
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
-20 Buyers
-8 Buyers
-36 Buyers
-12 Buyers
-6 Buyers
== 18 Buyers
Step 1: Identify the Biggest Problem
Use Funnel Metrics Quant
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability Qual
Problem Possibilities
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
-36 People
• Didn’t have payment info handy
• Confused by specific question
• Looked for promo code
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Create Solution Hypotheses
Solution Hypothesis
Problem
People looked for a promo code.
Solution Hypothesis
Possible Solution
Remove promo code entirely.
Solution Hypothesis
What Do We Think Would Happen?
An increase in conversion.
Solution Hypothesis
What Might Go Wrong?
A decrease in acquisition or revenue.
Step 4: Learn & Iterate*
*The last step is always learn & iterate!
Quant &
Qual
Learn…
How Would We Know We Were Right?
A/B test of the checkout flow with and without the
promo code.
What Really Happened?
Create Account
Select Plan
Payment Info
Address
Confirm
Thanks & Up Sell
Why Did It Happen?
Qual!
…And Iterate
Another Possible Solution
Create custom landing pages rather than promo
codes.
Step 1: Identify the Biggest Problem
Use Funnel Metrics
Step 2: Understand Why
Observational Usability
Step 3: Propose Solutions
Create Solution Hypotheses
Step 4: Learn & Iterate!
Or…
Step 1: Identify the Biggest Problem
Step 1: Identify the Biggest Problem
Interview Current Users
Qual
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Study Relevant Metrics
Quant
But Why?
Step 1: Identify the Biggest Problem
Interview Current Users
Step 2: Understand What
Study Relevant Metrics
Step 3: Propose Solutions
Create Solution Hypotheses
Step 1: Identify the Biggest Problem
Look atYour Sales Pipeline
Step 2: Understand Why
Interview People Who Said No
Step 3: Propose Solutions
Create Solution Hypotheses
Step 4: Learn & Iterate*
*The last step is always learn & iterate!
Once Again…
Quantitative Data
WHAT
Qualitative Data
WHY
Better Data
==
Better Products
Questions?
@lauraklein
laura@usersknow.com
Qual vs. Quant: Using data to Build Better Products

Qual vs. Quant: Using data to Build Better Products