Metrics That Matter
Juliana Méndez
juliana@saleside.co
10 YEARS YEARS IN GROWTH MARKETING • FOUNDER AT SALESIDE • STARTUP AND
CORPORATE CONSULTANT • INNOVATION MENTOR • DATA ADVOCATE
1. The Growth Concept
2. Make Metrics Your Friend
3. Goal and KPI Setting Strategies
4. Finding Your OMTM
5. Growth Experiments
6. Real Life Cases
1. The Growth Concept
Build it and they will come
Build it and they will come
Digital 

Marketing

Strategy
Growth 

Strategy
Growth vs. Digital Marketing
Multiply impact on your goals
along the entire way
Why Growth?
No data
no learnings
This is where
things fall apart
People love
this part
Everyone’s idea
is the best, right?
The Core of Growth
DATA PRODUCT
IDEAS
BUILDLEARN
MEASURE
2. Make Metrics Your Friend
• Measure your movement towards your
Business goals
• Know what you will become when you
grow up (before the money runs out)
• Keeps us honest and let’s us learn
• Stay ahead of competition
Analytics Helps
The Additional Power of Data
• Data “puking” & drowning in data
• Making things look good & lying to ourselves
• Failing to focus, measure the wrong thing
• Everyone in the teams optimises for something else
• Not understandable for everyone in the teams
• False positives and negatives
• Not changing anything based on data
Analytics Fails
Things you can measure, but don’t matter
really….
• Hits and page views
• # of visits
• Unique visitors
• Followers & fans
• Time on site
• Brand affinity & presence
Careful With Vanity Metrics
• Unstructured
• Hard to aggregate
• Anecdotal
• Warm and fuzzy

→ These explain the why
Qualitative Quantitative
• Numbers and statistics
• Hard facts
• Easier to analyse and aggregate
• Cold and hard
→ These explain the where, how
many and how often
Types: Qualitative vs. Quantitative
• Historical
• Reports how you are doing
• Outcome measures
• Typical financial KPIs
→ e.g. sales, net revenue and ROI
• Forward-looking
• Performance goals
• Number today that predicts tomorrow
→ e.g. pipeline, customer
satisfaction, new market growth,
innovation speed and conversion rate
Types: Lagging vs. Leading
Lagging Leading
• Increase activation rate
by 20%
Metric Types: Pros and Cons
Example Pros Cons
Relative

Percentage
Absolute
Percentage
Absolute
Number
• Increase conversion
rate from 30% to 40%
• Increase users per
month by 10K
• None
• Feels easier for team to
control
• More focused/stable
• Gets rid of denominator
• Tied more directly to
key output
• Can hide small goals
• Decrease in
denominator
• Decreases in
denominator
• Can be hard to
calculate vs baseline
(Seasonality etc.)
SOURGE: REFORGE
• Understandable
• Comparative
• Rate or a ratio
• Measures uniques, not totals only
• Causal relationship towards a goal
• Actionable: Behaviour Changing
A good metric
3. Goal and KPI Setting Strategies
The North Star Metrics
The North Star Metric is the single metric that best
captures the core value that your product
delivers to customers.
• Understand the main value of your business for your customers
• Try to quantify this value in a single number
• It’s not just marketing, it’s a team effort, and a common goal
Nights booked
Finding the North Star Metric
…breaks down North Star to the
drivers to be optimised right now.
5% more new visitors per day
3 repeat orders per month/user
$15 Avg. Unique Order volume
1.25 Viral coefficient
8% new activated customers
OMTM – One Metric That Matters
• Tells you which are the most important questions you have
• Forces you to draw a line in the sand and set the right goals
• Avoids “data puking” & focuses the entire company
• Inspires experimentation
• Tool that contributes to long term sustainable growth
5 Reasons to Use The North Star Metric and OMTM
4. Finding Your OMTM
Finding Your OMTM
What stage
are you in?
What business
are you in?
Do we offer the right solution for
the right problem to the right
people?
How do we increase success
more efficiently, how do we
double down?
Which are the right channels,
how do shorten the path to
success?
The Startup Lifecycle
Searching for product-market fit Search for repeatable, scaleable,
and profitable growth model
=

“Stacking the odds”
Scaling the business
(Doubling down)
Empathy Stickiness Virality Revenue Scale
Efficiency
CAC /
LTV Ratio
Do users have a great first
experience? 

(High activation rate?)
What does retention
look like? (>80%?)
High NPS Score (>5?)
Efficient Growth
process
Qualitative data
Traffic, users
CTR’s
Conversion
Rate
Optimisation
Acquisition
Net 

Revenue
Share Value
Lifecycle Metrics
Activation Retention Referral Acquisition Revenue
Searching for product-market fit Search for repeatable, scaleable,
and profitable growth model
=

“Stacking the odds”
Scaling the business
(Doubling down)
AARRR is not the the order in which you should optimise.
Ordered depends on the business model and the user
experience
B2C Free

RRAAR
B2C or B2B Freemium

RRRAA
Enterprise B2B

RRRAA
Standard Pirate Metrics

AARRR
Business - The AARRR Order
3
3
1
2
3
1
2
1
2
3
3
1
2
eCommerce SaaS App2-Sided Marketplace Media & UGC
• Conversion-rate, Cart
Abandonment

• Avg. transaction value,
CAC, CLV

• Buyer Reactivation

• Shares, NPS

• Email Opt-ins
• Inventory/Listings Ratio

• Liquidity, CAC, CLV

• Engagement, Shares,
SEM, NPS

• Avg. Transaction value,
Commissions/rates
• Engagement/Churn, MRR
Growth Rate

• MRR Churn, Revenue per
Account, Expansion/
Upselling

• Lead Velocity, Reactivation
Rate

• CLV, CAC
• Downloads, churn, DAU/
MAU

• App ratings, CAC,
Retention Rate

• Daily Sessions, Shares

• ARPDAU, CLV, CAC
• Traffic, visits, returns,
pages per visitor, Content
quality

• Ad Fill Rate, Revenue/
page or visitor, eCPM,

• Revenue per Segment,
Engagement, ratings,
comments

• Loyalty, Influencer Ident.,
Volume
Find Your North Star Metric
Your Business Model
SOURGE: LEAN ANALYTICS
5. Growth Experiments
Create a Growth Experiment
DRAW A LINE IN THE SAND
FIND A POTENTIAL
IMPROVEMENT
WITH DATA

FIND A
COMMONALITY
WITHOUT DATA

MAKE A GOOD
GUESS
DESIGN A TEST
MAKE CHANGES IN
PRODUCTION
CREATE HYPOTHESIS
MEASURE THE RESULTS
DID WE MOVE THE
NEEDLE?
YES
NO
PICK A KPI
NEXT
IMPROVEMENT
SUCCESS
PIVOT OR GIVE UP
CHANGE THE TEST
CHANGE THE
HYPOTHESIS
Execute Roadm
ap
&
Responsibilities
Goals
Identify
Opportunities
Hypothesis
& Design
Prioritise
Analyse
SOURGE: AVINASH KAUSHIK
Form an hypothesis
• What is the Problem you are solving and
for which target group?
• What is the observation and assumption?
• What is the proposed solution?
• Which KPI will be impacted and what is
the predicted outcome?
Growth Experiment Elements
+ =
+ ++
→ If proposed solution is successful, this KPI will increase by predicted outcome/amount,
because assumption based on observation and problem
Potential
(1-5)
Prioritisation Methodologies
PIE
ICE
REFORGE
Impact
(1-10)
Success
(Low–Med–High)
Importance
(1-5)
Confidence
(1-10)
Effort
(Low–Med–High)
Ease
(1-5)
Effort
(1-10)
Upside
(Low–Med–High)
+ +
x x
CUE Confidence Upside Easex x
Each of the criteria of your method is scored and the
average / multiplication is the score of the experiment
Prioritisation Methodologies
Confidence Upside Ease
How many people is
the campaign reaching
compared to
alternatives (1-10)
What is the minimum
and the maximum
result the experiment
can reach? (1-10)
What is the long-term
impact, how does it
compound over time?
(1-10)
Reach
Ceiling
Time
How long does it take
to reach results?
How much external
costs? Agencies,
Software, Content, ads
etc.
How much do you or
your team need to
invest for preparation
and running?
Time
Costs
Resources
New areas you never
worked on and don’t
know anything about
Some experience and
knowledge here but
not extremely confident
A lot of experience and
are confident based on
previous campaigns/
learnings
1-3
4-6
7-10
6. Real Life Cases
Started as group in Facebook for friends
10M users.
Metrics
• OMTM: User engagement
• KPI: No of active users; no of circles with
activity
• Current: > 80% circles without activity
• Question: who is most engaged?
Segmented users
• Highest Engagement = Mothers

Hypothesis
“Moms will join, relate and engage in a
community targeted at them more and improve
active circles and engaged users by X%?”
Circle of Moms
Circle of Moms
Results:
• Messages 50% longer
• 115% more likely to attach a
picture
• 110% more likely to converse in
thread
• 60% more likely to accept
invitations
Pivot
• Initial huge user base drop
• User base growth
ENOUGH TO GROW
FOCUS ON MORE ENGAGED
GROUP OF USERS
WITH DATA

FIND A COMMONALITY
PRODUCTION

COMPANY REBRANDING
HYPOTHESIS

MOMS ARE MORE ENGAGED
RESULTS

MUCH BETTER
DID WE MOVE THE NEEDLE?
YES
PICK A KPI
NEXT IMPROVEMENT
SUCCESS
Hypothesis
“A better targeted landing page design will
improve the revenue per visitor by X“
• KPI: Revenue per visitor
• Developed and A/B tested 3 design
variations
• Winner 41% increase in revenue per
visitor
• Not focus on conversion
Freemium Model (Payed plans)
Main KPI
• Payed churn < 5% (Paying users who
downgrade to free or cancel)
• Engagement (Product used in the last
month)
Answers Questions:
• Does Marketing messaging work?
• Is feature development progressing in the
right direction?
• Bugs
Launch Hypothesis
“Our target group wants to have a Browser
experience. Offering a improved browser
solution improves engagement by X”
→ Hypothesis invalidated through
browser experiment (no engagement)
Pivot
Adding mobile applications > Success
• Growth = holistic effort across product, marketing and
sales
• No growth or learnings without measuring data
• Measure the right thing: Avoid bad or vanity
metrics. The right metrics aligns all teams.
• Stage and business model determines your OMTM
• Fast and metric-focused experimenting accelerates
growth. Failed experiments can be wins too.
• A good hypothesis describes a potential solution to
a problem, predicts an outcome of an experiment and
is prioritised before execution.
Business Analytics as a driver for innovation
Learnings and validation

of assumptions
Innovation through

uncovering patterns
Long-term success
→→
Books
• Lean Analytics
• Hacking Growth
• Web Analytics 2.0
Blogs
• Avinash Kaushik
• Andrew Chen
Articles
• Dave McClure – Startup Metrics for Pirates
• Avinash Kaushik – Lean Analytics Cycle
• a16z – 16 Startup Metrics
• a16z – 16 More Startup Metrics
Recommended Reading

ESADE Business Analytics Master: Metrics that matter

  • 1.
    Metrics That Matter JulianaMéndez juliana@saleside.co 10 YEARS YEARS IN GROWTH MARKETING • FOUNDER AT SALESIDE • STARTUP AND CORPORATE CONSULTANT • INNOVATION MENTOR • DATA ADVOCATE
  • 2.
    1. The GrowthConcept 2. Make Metrics Your Friend 3. Goal and KPI Setting Strategies 4. Finding Your OMTM 5. Growth Experiments 6. Real Life Cases
  • 3.
  • 4.
    Build it andthey will come
  • 5.
    Build it andthey will come
  • 6.
  • 7.
    Multiply impact onyour goals along the entire way Why Growth?
  • 8.
    No data no learnings Thisis where things fall apart People love this part Everyone’s idea is the best, right? The Core of Growth DATA PRODUCT IDEAS BUILDLEARN MEASURE
  • 9.
    2. Make MetricsYour Friend
  • 10.
    • Measure yourmovement towards your Business goals • Know what you will become when you grow up (before the money runs out) • Keeps us honest and let’s us learn • Stay ahead of competition Analytics Helps
  • 11.
  • 12.
    • Data “puking”& drowning in data • Making things look good & lying to ourselves • Failing to focus, measure the wrong thing • Everyone in the teams optimises for something else • Not understandable for everyone in the teams • False positives and negatives • Not changing anything based on data Analytics Fails
  • 13.
    Things you canmeasure, but don’t matter really…. • Hits and page views • # of visits • Unique visitors • Followers & fans • Time on site • Brand affinity & presence Careful With Vanity Metrics
  • 14.
    • Unstructured • Hardto aggregate • Anecdotal • Warm and fuzzy
 → These explain the why Qualitative Quantitative • Numbers and statistics • Hard facts • Easier to analyse and aggregate • Cold and hard → These explain the where, how many and how often Types: Qualitative vs. Quantitative
  • 15.
    • Historical • Reportshow you are doing • Outcome measures • Typical financial KPIs → e.g. sales, net revenue and ROI • Forward-looking • Performance goals • Number today that predicts tomorrow → e.g. pipeline, customer satisfaction, new market growth, innovation speed and conversion rate Types: Lagging vs. Leading Lagging Leading
  • 16.
    • Increase activationrate by 20% Metric Types: Pros and Cons Example Pros Cons Relative
 Percentage Absolute Percentage Absolute Number • Increase conversion rate from 30% to 40% • Increase users per month by 10K • None • Feels easier for team to control • More focused/stable • Gets rid of denominator • Tied more directly to key output • Can hide small goals • Decrease in denominator • Decreases in denominator • Can be hard to calculate vs baseline (Seasonality etc.) SOURGE: REFORGE
  • 17.
    • Understandable • Comparative •Rate or a ratio • Measures uniques, not totals only • Causal relationship towards a goal • Actionable: Behaviour Changing A good metric
  • 18.
    3. Goal andKPI Setting Strategies
  • 19.
    The North StarMetrics The North Star Metric is the single metric that best captures the core value that your product delivers to customers.
  • 20.
    • Understand themain value of your business for your customers • Try to quantify this value in a single number • It’s not just marketing, it’s a team effort, and a common goal Nights booked Finding the North Star Metric
  • 21.
    …breaks down NorthStar to the drivers to be optimised right now. 5% more new visitors per day 3 repeat orders per month/user $15 Avg. Unique Order volume 1.25 Viral coefficient 8% new activated customers OMTM – One Metric That Matters
  • 22.
    • Tells youwhich are the most important questions you have • Forces you to draw a line in the sand and set the right goals • Avoids “data puking” & focuses the entire company • Inspires experimentation • Tool that contributes to long term sustainable growth 5 Reasons to Use The North Star Metric and OMTM
  • 23.
  • 24.
    Finding Your OMTM Whatstage are you in? What business are you in?
  • 25.
    Do we offerthe right solution for the right problem to the right people? How do we increase success more efficiently, how do we double down? Which are the right channels, how do shorten the path to success? The Startup Lifecycle Searching for product-market fit Search for repeatable, scaleable, and profitable growth model =
 “Stacking the odds” Scaling the business (Doubling down) Empathy Stickiness Virality Revenue Scale
  • 26.
    Efficiency CAC / LTV Ratio Dousers have a great first experience? (High activation rate?) What does retention look like? (>80%?) High NPS Score (>5?) Efficient Growth process Qualitative data Traffic, users CTR’s Conversion Rate Optimisation Acquisition Net Revenue Share Value Lifecycle Metrics Activation Retention Referral Acquisition Revenue Searching for product-market fit Search for repeatable, scaleable, and profitable growth model =
 “Stacking the odds” Scaling the business (Doubling down)
  • 27.
    AARRR is notthe the order in which you should optimise. Ordered depends on the business model and the user experience B2C Free RRAAR B2C or B2B Freemium RRRAA Enterprise B2B RRRAA Standard Pirate Metrics AARRR Business - The AARRR Order 3 3 1 2 3 1 2 1 2 3 3 1 2
  • 28.
    eCommerce SaaS App2-SidedMarketplace Media & UGC • Conversion-rate, Cart Abandonment • Avg. transaction value, CAC, CLV • Buyer Reactivation • Shares, NPS • Email Opt-ins • Inventory/Listings Ratio • Liquidity, CAC, CLV • Engagement, Shares, SEM, NPS • Avg. Transaction value, Commissions/rates • Engagement/Churn, MRR Growth Rate • MRR Churn, Revenue per Account, Expansion/ Upselling • Lead Velocity, Reactivation Rate • CLV, CAC • Downloads, churn, DAU/ MAU • App ratings, CAC, Retention Rate • Daily Sessions, Shares • ARPDAU, CLV, CAC • Traffic, visits, returns, pages per visitor, Content quality • Ad Fill Rate, Revenue/ page or visitor, eCPM, • Revenue per Segment, Engagement, ratings, comments • Loyalty, Influencer Ident., Volume Find Your North Star Metric Your Business Model SOURGE: LEAN ANALYTICS
  • 29.
  • 30.
    Create a GrowthExperiment DRAW A LINE IN THE SAND FIND A POTENTIAL IMPROVEMENT WITH DATA FIND A COMMONALITY WITHOUT DATA MAKE A GOOD GUESS DESIGN A TEST MAKE CHANGES IN PRODUCTION CREATE HYPOTHESIS MEASURE THE RESULTS DID WE MOVE THE NEEDLE? YES NO PICK A KPI NEXT IMPROVEMENT SUCCESS PIVOT OR GIVE UP CHANGE THE TEST CHANGE THE HYPOTHESIS Execute Roadm ap & Responsibilities Goals Identify Opportunities Hypothesis & Design Prioritise Analyse SOURGE: AVINASH KAUSHIK
  • 31.
    Form an hypothesis •What is the Problem you are solving and for which target group? • What is the observation and assumption? • What is the proposed solution? • Which KPI will be impacted and what is the predicted outcome? Growth Experiment Elements + = + ++ → If proposed solution is successful, this KPI will increase by predicted outcome/amount, because assumption based on observation and problem
  • 32.
  • 33.
    Prioritisation Methodologies Confidence UpsideEase How many people is the campaign reaching compared to alternatives (1-10) What is the minimum and the maximum result the experiment can reach? (1-10) What is the long-term impact, how does it compound over time? (1-10) Reach Ceiling Time How long does it take to reach results? How much external costs? Agencies, Software, Content, ads etc. How much do you or your team need to invest for preparation and running? Time Costs Resources New areas you never worked on and don’t know anything about Some experience and knowledge here but not extremely confident A lot of experience and are confident based on previous campaigns/ learnings 1-3 4-6 7-10
  • 34.
  • 35.
    Started as groupin Facebook for friends 10M users. Metrics • OMTM: User engagement • KPI: No of active users; no of circles with activity • Current: > 80% circles without activity • Question: who is most engaged? Segmented users • Highest Engagement = Mothers Hypothesis “Moms will join, relate and engage in a community targeted at them more and improve active circles and engaged users by X%?” Circle of Moms
  • 36.
    Circle of Moms Results: •Messages 50% longer • 115% more likely to attach a picture • 110% more likely to converse in thread • 60% more likely to accept invitations Pivot • Initial huge user base drop • User base growth ENOUGH TO GROW FOCUS ON MORE ENGAGED GROUP OF USERS WITH DATA FIND A COMMONALITY PRODUCTION COMPANY REBRANDING HYPOTHESIS MOMS ARE MORE ENGAGED RESULTS MUCH BETTER DID WE MOVE THE NEEDLE? YES PICK A KPI NEXT IMPROVEMENT SUCCESS
  • 37.
    Hypothesis “A better targetedlanding page design will improve the revenue per visitor by X“ • KPI: Revenue per visitor • Developed and A/B tested 3 design variations • Winner 41% increase in revenue per visitor • Not focus on conversion
  • 38.
    Freemium Model (Payedplans) Main KPI • Payed churn < 5% (Paying users who downgrade to free or cancel) • Engagement (Product used in the last month) Answers Questions: • Does Marketing messaging work? • Is feature development progressing in the right direction? • Bugs
  • 39.
    Launch Hypothesis “Our targetgroup wants to have a Browser experience. Offering a improved browser solution improves engagement by X” → Hypothesis invalidated through browser experiment (no engagement) Pivot Adding mobile applications > Success
  • 40.
    • Growth =holistic effort across product, marketing and sales • No growth or learnings without measuring data • Measure the right thing: Avoid bad or vanity metrics. The right metrics aligns all teams. • Stage and business model determines your OMTM • Fast and metric-focused experimenting accelerates growth. Failed experiments can be wins too. • A good hypothesis describes a potential solution to a problem, predicts an outcome of an experiment and is prioritised before execution. Business Analytics as a driver for innovation Learnings and validation
 of assumptions Innovation through
 uncovering patterns Long-term success →→
  • 41.
    Books • Lean Analytics •Hacking Growth • Web Analytics 2.0 Blogs • Avinash Kaushik • Andrew Chen Articles • Dave McClure – Startup Metrics for Pirates • Avinash Kaushik – Lean Analytics Cycle • a16z – 16 Startup Metrics • a16z – 16 More Startup Metrics Recommended Reading