The document discusses the importance of defining an analytics strategy for startups. It explains that analytics can help startups learn faster through feedback loops, provide more clarity on customer behavior, and build consensus on future actions. The document then outlines keys to a great analytics strategy, including tightly integrating analytics with business strategy, using an iterative process, and defining measurable hypotheses. It recommends focusing analytics on segmentation/cohort analysis, retention, funnels, revenue tracking, and marketing effectiveness. Case studies on Airbnb, Khan Academy, and Jawbone demonstrate how analytics provided insights to optimize processes and better understand customer needs.

1. Lean LaunchPad Workshop:
Defining an Analytics Strategy
Ryan Jung
Haas MBA 2014
ryan_jung@haas.berkeley.edu

2. Why Are Analytics Important?
• Failure to define an
analytics strategy can be
a fatal error for a startup
in 2015.
• Analytics has changed
the landscape
• A great analytics
strategy is tightly
integrated with the
overall business strategy

3. Why You Need an Analytics Strategy
• Learn faster by
creating
feedback loops
• More clarity
based on
behavior
• Consensus on
future action
There exists a host of tools to help you with
these objectives.

4. History of Analytics
• 1990s – Web counters
• 2000s – Click Analytics
and SEO
• 2010s – Behavioral
and Predictive
Analytics

5. Keys to a Great Analytics Strategy
1. Tightly integrated
with overall
business strategy
2. Iterative process
3. Measurable set of
hypotheses,
results, and
revisions

6. The Modern Data-Driven Lean Startup
Goal is to optimize a set of
business objectives in a logical
progression leveraging quantitative
and qualitative facts in order to
delight customers in a scalable,
repeatable fashion

7. Most Important Reports
• Segmentation (Cohorting)
• Retention
• Funnels
• Revenue Tracking
• Marketing Campaign Effectiveness
• Path Analysis
• Notifications

8. Segmentation / Cohorting
What segments are getting what
value out of your product?

9. Value Proposition / Customer Segment
Who is our customer?
What problem are we really
solving for them?
Will they buy from us?
How do we reach them?
• Build customer archetypes
• Add properties to define the user
• Use segmentation to look at differences in customers
• Good for looking at actions, but need to understand causation to be actionable
Using Analytics

10. Segmentation Example
• Look at aggregated
events and then
segment by properties
• See who is doing
particular actions and
identify trends
• Want to segment as far
as possible
• Point you to needs and
how your product adds
value
Google Analytics

11. Retention
Who gets the most value out of your
solution?

12. How Churn affects LTV
Lifetime Value
Monthly
Churn
Source: David Skok Matrix Partners

13. Thinking Through Retention
Get –> Keep –> Grow = Activation –>
Retention –> Engagement
Understanding key features
Understanding core users and testing
their needs
Identifying most effective channels

14. Retention Reports
In-session retention In-app retention
Key Question(s) Where do users spend their time in
your app? What features are
valuable?
Are users coming back and using the
app repeatedly? Who are users that
are more likely to come back?
Value Proposition Features that are most valuable Users that get most value out of
product
Tool Addiction Recurring or Segmented Retention
Mixpanel

15. BIG IDEA:
LTV drives CAC which drives channel
selection
Increasing Sales Complexity
Log(AcquisitionCost)
CAC < LTV

16. Funnels
How are users interacting with your
solution?

17. Sales Funnels
Where are we losing
customers?
How do we know if we
are doing well or not
well in sales?
How can we do better?
Core Idea: Track conversion rates between
levels of funnel to see where “leakage” occurs
and create strategies to minimize this loss.
Is my marketing spend
being used efficiently?

18. Funnel Reports
Localytics

19. Funnel Reports
KISSMetrics

20. Tying funnels to revenues
Revenue = installs x [signups / installs] x [purchases / signups ] x [revenue / purchase]
Back-end tells
you this
Analytics tells
you this
Analytics can
tell you this
You control this
The main point here is that you can break revenue into measureable
components
• Tie how you earn revenue to what you measure
• Then understand where you are doing well and not well
• Then use your analytics solution to design tests to figure out how to drive
more lifetime value
Mathematically:

21. Pitfalls to Avoid
Problem Explanation
Search vs. Execution
Metrics
Are we measuring KPIs or are we testing
hypotheses?
Vanity metrics If it only goes “up and to the right” and / or
if it’s not actionable, it’s a waste of time to
measure it.
Biased tests Be sure that the hypotheses that you are
testing are not set up to confirm your
assumptions. Take the approach of trying to
disprove your hypothesis.
Data overload “Measuring everything and then mining for
insights” creates too much noise for most to
get any real value from.

22. Summary
• You need to be thinking about analytics
because your competition probably already is
• Analytics is evolving, so keeping up is
imperative
• Analytics needs to be tied to your overall
business strategy, should be hypothesis-
driven, and is an iterative process

23. Case Studies

24. Airbnb
• Challenge: Initially wanted to optimize booking flow
• Allowed them to identify to distinct classes of users
• Can better target users and their needs
More info: https://mixpanel.com/case-study/airbnb/

25. Khan Academy
• Challenge: increase engagement and the rate at which people
learn
• Used funnels to optimize search and registration processes
• Start with a definition for “user engagement”
More info: https://mixpanel.com/case-study/khanacademy/

26. Jawbone
• Challenge: Assess the viability of Jawbone UP
• Used Segmentation reporting to better understand their users
• Helps to build customer archetypes
• Faster iterations and faster time to product-market fit
More info: https://mixpanel.com/case-study/jawbone/

27. Cohort analysis
Renewal and upsell rates
Return on marketing investment

28. Revenue by Cohort – Each Year Builds on a Stronger Base
Note: Excludes inorganic growth.
201120102009200820072006
Highly Loyal Customers
2007 Cohort
Earlier Cohorts

29. Revenue by Cohort – Each Year Builds on a Stronger Base
2006
2008 Cohort
2009 Cohort
2010 Cohort
2011 Cohort
20112010200920082007
Highly Loyal Customers
Note: Excludes inorganic growth.
2007 Cohort
Earlier Cohorts