SlideShare a Scribd company logo
1 of 141
Download to read offline
Lean Analytics
DCU
April, 2014
@acroll
Some housekeeping.
Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.
The core of Lean
is iteration.
Most startups don’t know what they’ll
be when they grow up.
Hotmail
was a
database
company
Flickr
was going to
be an MMO
Twitter
was a
podcasting
company
Autodesk
made
desktop
automation
Paypal
first built for
Palmpilots
Freshbooks
was invoicing
for a web
design firm
Wikipedia
was to be
written by
experts only
Mitel
was a
lawnmower
company
Unfortunately,
we’re all liars.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall apart.
No data, no
learning.
Analytics can help.
Analytics is the measurement of
movement towards your business
goals.
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
I have two kids.
At least one of them is a girl.
What are the chances
the other is a boy?
BB BG
GB GG
2 of 3 (66%) are boys.
GB GG BG
Some fundamentals.
A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Comparison is
context.
A ratio or rate
The only way to
measure
change and roll
up the tension
between two
metrics (MPH)
Behavior
changing
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
The
simplest
rule
bad
metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%
of retailers
20%
of retailers
10%
of retailers
You are
just like
Customers that
buy >1x in 90d
Once
2-2.5
per year
>2.5
per year
Your customers
will buy from you
Then you are
in this mode
1-15%
15-30%
>30%
Low acquisition
cost, high checkout
Increasing return
rates, market share
Loyalty, selection,
inventory size
Focus on
(Thanks to Kevin Hillstrom for this.)
Qualitative
Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.
Warm and fuzzy.
Quantitative
Numbers and stats.
Hard facts, less insight,
easier to analyze; often
sour and disappointing.
Cold and hard.
Exploratory
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Cool.
Reporting
Predictable. Keeps you
abreast of the normal,
day-to-day operations.
Can be managed by
exception.
Necessary.
Rumsfeld on Analytics
(Or rather, Avinash Kaushik channeling Rumsfeld)
Things we
know
don’t
know
we know Are facts which may be wrong and
should be checked against data.
we don’t
know
Are questions we can answer by
reporting, which we should baseline
& automate.
we know
Are intuition which we should
quantify and teach to improve
effectiveness, efficiency.
we don’t
know
Are exploration which is where
unfair advantage and interesting
epiphanies live.
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this is the April cohort)
A/B test:
Changing one thing
(i.e. color) and
measuring the
result (i.e. revenue.)
Multivariate
analysis
Changing several
things at once to
see which correlates
with a result.
☀
☁
☀
☁
Segment:
Cross-sectional
comparison of all
people divided by
some attribute (age,
gender, etc.)
☀
☁
Which of these two companies
is doing better?
  January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohort 1 2 3 4 5
January
February
March
April
May
$5 $3 $2 $1 $0.5
$6 $4 $2 $1
$7 $6 $5
  $8 $7
      $9
How about
this one?
Cohort 1 2 3 4 5
January
February
March
April
May
Averages
$5 $3 $2 $1 $0.5
$6 $4 $2 $1  
$7 $6 $5    
$8 $7      
$9        
$7 $5 $3 $1 $0.5
Look at the
same data
in cohorts
Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-looking.
Number today that
predicts tomorrow;
reports the news.
Example: pipeline.
Predicting the
future.
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a day after signing up for a game,
they’ll probably become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device
(ChenLi Wang)
Twitter user following a certain number of people, and a certain
percentage of those people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
Some examples
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk
about correlation.
1
10
100
1000
10000
Ice cream consumption Drownings
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An independent variable
that directly impacts a
dependent one.
Summertime &
drowning.
A leading, causal metric
is a superpower.
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Growth hacking, demystified.
Find
correlation
Test
causality
Optimize the
causal factor
Pick a metric
to change
Is social action a leading
indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Is mobile use?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Why is Nigerian spam so badly
written?
Aunshul Rege of Rutgers University, USA in 2009
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages
emailed; they expect to land 2 or 3 “Mugu” (fools) each week.
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money”
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts
the true to false positive ratio in his favor.”
1000 emails
1-2 responses
1 fool and their money, parted.
Bad language (0.1% conversion)
Gullible (70% conversion)
1000 emails
100 responses
1 fool and their money, parted.
Good language (10% conversion)
Not-gullible (.07% conversion)
This would be horribly
inefficient since
humans are involved.
Turns out the word “Nigeria” is the best
way to identify promising prospects.
Nigerian spammers
really understand their target market.
They see past vanity metrics.
The Lean Analytics framework.
Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spend money to
get customers.
Customers are
worth more than
they cost.
Stickiness
Keep people
coming back.
Approach
Get customers
faster than you
lose them.
Math that
matters
Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, blogs ...
Activation
Do drive-by visitors subscribe, use, etc?
Features, design, tone, compensation, affirmation ...
Retention
Does a one-time user become engaged?
Notifications, alerts, reminders, emails, updates...
Revenue
Do you make money from user activity?
Transactions, clicks, subscriptions, DLC, analytics...
Referral
Do users promote your product?
Email, widgets, campaigns, likes, RTs, affiliates...
Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve the problem in a
way they will keep using and pay for.
VIRALITY
I’ve found ways to get them to tell their friends,
either intrinsically or through incentives.
REVENUE
The users and features fuel growth organically
and artificially.
SCALE
I’ve found a sustainable, scalable business with
the right margins in a healthy ecosystem.
Gate
Thefivestages
Empathy stage:
Localmind hacks Twitter
Needed to find out if a core assumption—strangers answering
questions—was valid.
Ran Twitter experiment instead of writing code
Asked senders of geolocated Tweets from Times Square random
questions; counted response rate
Conclusion: high enough to proceed
Stickiness stage:
qidiq streamlines invites
Survey owner adds recipient to group
Survey owner asks question
Recipient reads survey question
Recipient responds to question
Recipient sees survey results
(Later, if needed…)
Recipient visits site; no password!
Recipient does password recovery
One-time link sent to email
Recipient creates password
Recipient can edit profile, etc.
Survey owner adds recipient to group
Survey owner asks question
Recipient gets invite
Recipient reads survey question
Recipient responds to question
Recipient installs mobile app
Recipient creates account, profile
Recipient sees survey results
Recipient can edit profile, etc.
10-25%RESPONSERATE
70-90%RESPONSERATE
Six business model archetypes
(Yours is probably a blend of these.)
E-commerce
SaaS (freemium?)
Mobile app (gaming)
Two sided marketplace
Media
User generated content
(Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/trial offer
Enrollment
User
Disengaged User
Cancel
Freemium
churn
Engaged User
Free user
disengagement
Reactivate
Cancel
Trial abandonment
rate
Invite Others
Paying Customer
Reactivation
rate
Paid
conversion
FORMER USERS
User Lifetime Value
Reactivate
FORMER CUSTOMERS
Customer Lifetime Value
Viral coefficient
Viral rate
Resolution
Support data
Account Cancelled Billing Info Exp.
Paid Churn Rate
Tiering
Capacity Limit
Upselling
rate Upselling
Disengaged DissatisfiedTrial Over
Model + Stage = One Metric That Matters.
One Metric
That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media UCG
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.
Was a marketing campaign successful?
Were customer complaints lowered?
Was a product upgrade valuable?
Net adds up:
Can we acquire more valuable customers?
What product features can increase engagement?
Can we improve customer support?
Net adds flat:
Are the new customers not the right segment?
Did a marketing campaign fail?
Did a product upgrade fail somehow?
Is customer support falling apart?
Net adds down:
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile
app
User-gen
content
2-sided
market
Interviews; qualitative results; quantitative scoring; surveys
Loyalty,
conversion
CAC, shares,
reactivation
Transaction,
CLV
Affiliates,
white-label
Engagement,
churn
Inherent
virality, CAC
Upselling,
CAC, CLV
API, magic #,
mktplace
Content,
spam
Invites,
sharing
Ads,
donations
Analytics,
user data
Inventory,
listings
SEM, sharing
Transactions,
commission
Other
verticals
(Money from transactions)
Downloads,
churn, virality
WoM, app
ratings, CAC
CLV,
ARPDAU
Spinoffs,
publishers
(Money from active users)
Traffic, visits,
returns
Content
virality, SEM
CPE, affiliate
%, eyeballs
Syndication,
licenses
(Money from ad clicks)
Better: bit.ly/BigLeanTable
What other metrics
do you want to know about?
Drawing some lines in the sand.
A company loses a quarter of its
customers every year.
Is this good or bad?
Not knowing what normal is
makes you do stupid things.
Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doing exceptionally well. If you
can only manage 1%, it's a sign
you haven't yet figured out
what you're doing.” At revenue
stage, measure growth in
revenue. Before that, measure
growth in active users.
Paul Graham, Y Combinator
• Are there enough people who really care
enough to sustain a 5% growth rate?
• Don’t strive for a 5% growth at the expense
of really understanding your customers
and building a meaningful solution
• Once you’re a pre-revenue startup at or
near product/market fit, you should have
5% growth of active users each week
• Once you’re generating revenues, they
should grow at 5% a week
Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day use web or mobile app
1% of users/day use it concurrently
Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5% monthly churn rate before you know you’ve got a
business that’s ready to grow (Mark MacLeod) and around 2%
before you really step on the gas (David Skok)
• Last-ditch appeals and reactivation can have a big impact.
Facebook’s “don’t leave” reduces attrition by 7%.
Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits
Baseline:
Calculating customer lifetime
25%
monthly churn
100/25=4
The average
customer lasts
4 months
5%
monthly churn
100/5=20
The average
customer lasts
20 months
2%
monthly churn
100/2=50
The average
customer lasts
50 months
Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time to find
out whether your churn and revenue projections
are right
• Cashflow: You’re basically “loaning” the
customer money between acquisition and CLV.
• It keeps you honest: Limiting yourself to a
CAC of only a third of your CLV will forces you
to verify costs sooner.
Lifetime of 20 mo.
$30/mo. per
customer
$600 CLV
$200 CAC
Now segment
those users!
1/3 spend
The Lean Analytics cycle
Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
Do AirBnB hosts
get more business
if their property is
professionally
photographed?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers posing as employees
Measure the results
Compare photographed listings to a control group
Make a decision
Launch photography as a new feature for all hosts
5,000 shoots per month
by February 2012
Hang on a second.
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?
Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
“Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses have in
common?”
Camera model.
With data:
find a commonality
Without data: make a
good guess
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publisher analytics
SaaS analytics
Gaming analytics
User interaction Customer satisfaction KPI dashboardsUser segmentation
User analytics Spying on users
Some non-tech examples.
I lied. Everyone is a tech company.
http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326
Cost of experiments: down. Cost of attention: way up.
Let’s pick on restaurants
for a while.
A line in the sand
Labor costs
Gross revenue
30%
24%
20%
Too costly?
Just right
Understaffed?
=
A leading indicator
http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970
50 reservations
at 5PM
250 covers
that night
(Varies by
restaurant.
McDonalds
≠ Fat Duck.)
http://www.flickr.com/photos/southbeachcars/6892880699
Restaurant MVP
Is tip amount a leading indicator of long-
term revenue?
Why does every table get the same
menu?
Is purple ink better?
http://tippingresearch.com/uploads/managing_tips.pdf
Lean Analytics workshop for Dublin City University, April 2014
Growth hacking
(is a word you should hate but will hear a lot about.)
Growth hacking, demystified.
Find
correlation
Test
causality
Optimize the
causal factor
Pick a metric
to change
Guerrilla
marketing
Data-
driven
learning
Subversiveness
GROWTH
HACKING
A Facebook user reaching 7 friends within 10 days of signing up (Chamath
Palihapitiya)
If someone comes back to Zynga a day after signing up for a game, they’ll probably
become an engaged, paying user (Nabeel Hyatt)
A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang)
Twitter user following a certain number of people, and a certain percentage of those
people following the user back (Josh Elman)
A LinkedIn user getting to X connections in Y days (Elliot Schmukler)
(from the 2012 Growth Hacking conference) (These are also great segments to analyze.)
The leading indicator
• Growth hacking is simply what marketing should have been
doing, but it fell in love with Don Draper and opinions along the
way
• Optimize a factor you think is correlated with growth
The growth hack
AirBnB and Craigslist
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
What if you’re in
a big organization?
The Zero Overhead principle
A central theme to this new wave of innovation is the
application of core product tenets from the consumer space
to the enterprise.
In particular, a universal lesson that I keep sharing with all
entrepreneurs building for the enterprise is the Zero
Overhead Principle: no feature may add training costs to
the user.
DJ Patil
The B2B stereotype
http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html
Domain
expert
Disruption
expert
•Domain expert knows industry and
the problem domain. Has a Rolodex;
proxy for customers.
•Disruption expert knows tech that will
produce a change Sees beyond the
current model.
The Lean Analytics lifecycle of an Intrapreneur
Empathy Consulting to test ideas and
bootstrap the business
Lock-in, IP control,
overfitting
Stickiness Standardization and integration;
shift from custom to generic
Ability to integrate;
support
Virality Word of mouth, references, case
studies
Bad vibes; exclusivity
Revenue Growing direct sales, professional
services, support
Pipeline, revenue
recognition, comp
Scale Channels, analysts, ecosystems,
APIs, vertically targeted products
Crossing the chasm;
Gorillas
Stage Do this Fear this
Enterprise-focused startups:
Metrics that matter
• Ease of customer engagement
and feedback
• Alpha/beta pipeline
• Stickiness and usability
• Integration costs
• User engagement with the app
• Disentanglement after a sale
• Support costs
• User groups and feedback
• Pitch success for channel tools
• Barriers to exit
What if you’re in
a big organization?
If a startup is an organization designed
to search for a sustainable, repeatable
business model, then an established
company is an organization designed
to perpetuate one.
The Lean Analytics lifecycle of an Intrapreneur
Empathy
Find problems; don’t test demand.
Skip the business case, do analytics
Entitled, aggrieved
customers
Stickiness
Know your real minimum based on
expectations, regulations
Hidden “must haves”,
feature creep
Virality
Build inherent virality in from the
start; attention is the new currency
Luddites who don’t
understand sharing
Revenue
Consider the ecosystem, channels,
and established agreements
Channel conflict,
resistance, contracts
Scale
Hand the baton to others gracefully Hating what happens
to your baby
Beforehand
Get buy-in Political fallout
Some things that work.
Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organization still learns.
http://www.flickr.com/photos/creative_tools/8544475139
Transformative isolation:
Skunkworks
Use outliers and missed searches to
hunt for good ideas & adjacencies
(Multi-billion-dollar hygiene product company)
1/8 men have an incontinence issue. 1/3
women do.
When search results show a significant
number of men searching, this suggests the
adjacent (male) market is underserved.
Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
Once the value was obvious to
management, got license to dig.
Focus on the desired behavior, not just
the information.
http://www.psychologytoday.com/blog/yes/
200808/changing-minds-and-changing-towels
26% increase in towel
re-use with an appeal
to social norms; 33%
increase when tied to
the specific room.
Energy Conservation “Nudges” and Environmentalist
Ideology: Evidence from a Randomized Residential Electricity
Field Experiment - Costa & Kahn 2011
The effectiveness of energy
conservation “nudges” depends on
an individual’s political ideology ...
Conservatives who learn that their
consumption is less than their
neighbors’ “boomerang” whereas
liberals reduce their consumption.
Understand hidden
constraints
That pencil story is a myth.
Graphite is conductive and
explosive. The Minimum
Viable Product is Viable for a
reason.
Know what has to
be built in-house
SAP integration
Employment law
Run it as a consulting business first.
(Just don’t get addicted to it. Your goal is to
learn and overcome integration challenges and
find the 20% of features that 80% of the market
will pay for.)
When in doubt, collect data
From tackling the FTA rate to
visualizing the criminal justice
supply chain.
Everything’s an excuse to experiment
Find other ways
to collect data;
everything is an
experiment.
Don’t just collect data,
chase it.
Some tools and traps
Traction graphs
Your business model
The stage you’re at
Your one metric
... change often if
you’re doing it right.
So how do you track
that over time?
Traction graphs
Jan Feb Mar Apr May Jun
Signups
per day
Conversion
rate
Churn
rate
Viral
coefficient
This axis changes for
each metric
Traction graphs
Jan Feb Mar Apr May Jun
Signups
per day
Conversion
rate
Churn
rate
Viral
coefficient
0%
Use vanity to get to
meaningful metrics
Your goal is to produce
outcomes
If the outcomes require
action, and vanity motivates
actors, use it
But show how the vanity
metric is a leading indicator of
the real one
x
Web traffic
Revenue
Activation
Cart
Size
Conversion
rate
The three threes
Three
assumptions
What big bets are you making?
•“People will answer questions”
•“Organizers are frustrated with how to run conferences”
•“We'll make money from parents”
•“Amazon is reliable enough for our users.”
Three actions
to take
What are you doing to make these assumptions happen (or
identify they’re wrong and change course?)
•Product enhancements
•Marketing strategies
Three experiments
to run
•Feature tests
•Continuous deployment
•A/B testing
•Customer survey
The three threes
Three
assumptions
Three actions
to take
Three experiments
to run
Monthly
Weekly
Daily
Board, investors,
founders
Executive team
Employees
Strategy
Tactics
Execution
The three threes
Three
assumptions
Three actions
to take
Three experiments
to run
Get more
people
Increase
answer %
Test better
questions
Change
the UI
Test
timings
Questions
from peers
Many people will
answer questions
The problem-solution canvas
CURRENT STATUS
• List key metrics you’re
tracking, where they’re at, and
compare with last few weeks
• How are things trending?
LAST WEEK’S LESSONS LEARNED
AND ACCOMPLISHMENTS)
• What did you learn last week?
• What was accomplished?
• On track: YES / NO?
The Goal is to Learn
The problem-solution canvas
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start
working on next week. Rank them.
• Why do you believe each solution will
help you solve or complete solve the
problem?
METRICS / PROOF + GOALS
Problem #1 (put name here)
• Metrics you’ll use to measure whether
or not the solutions are doing what you
hoped (solving the problem)
• List proof (qualitative) you’ll use as well
• Define goals for the metric
HYPOTHESIZED SOLUTIONS
• List possible solutions that you’ll start
working on next week. Rank them.
METRICS / PROOF + GOALS
• Metrics you’ll use to measure whether
or not the solutions are doing what you
hoped (solving the problem)
Problem #2 (put name here)
“The most important figures that one
needs for management are unknown
or unknowable, but successful
management must nevertheless take
account of them.”
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others
in the absence of data.
Now, a leader knows
what questions to ask.
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko
Follow-on:
The other business model
system diagrams
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
The mobile app!
customer lifecycle!
Ratings
Reviews
Search
Leaderboards
Purchases
Downloads
Installs
Play
Disengagement
Reactivation
Uninstallation
Disengagement
Account"
creation
Virality
Downloads,"
Gross revenue
ARPU
App sales
Activation
Churn, CLV
In-app"
purchases
Appstore!
Incentivized
Legitimate
Fraudulent
Ratings!
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014

More Related Content

What's hot

B2B SaaS north star metric 2019 2020
B2B SaaS north star metric 2019 2020B2B SaaS north star metric 2019 2020
B2B SaaS north star metric 2019 2020Gokul Rangarajan
 
Lean Analytics Cycle
Lean Analytics CycleLean Analytics Cycle
Lean Analytics CycleHiten Shah
 
Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassLean Analytics
 
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...saastr
 
Basics of AB testing in online products
Basics of AB testing in online productsBasics of AB testing in online products
Basics of AB testing in online productsAshish Dua
 
What's a Growth Stack? And why you should build one.
What's a Growth Stack? And why you should build one. What's a Growth Stack? And why you should build one.
What's a Growth Stack? And why you should build one. HubSpot
 
Google analytics 4 - présentation détaillée
Google analytics 4 - présentation détailléeGoogle analytics 4 - présentation détaillée
Google analytics 4 - présentation détailléeBruno Guyot
 
Growth Hacking Workshop: AAU
Growth Hacking Workshop: AAUGrowth Hacking Workshop: AAU
Growth Hacking Workshop: AAUMark Andersen
 
Causality without headaches
Causality without headachesCausality without headaches
Causality without headachesBenoît Rostykus
 
Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Lean Analytics
 
傾向スコアを使ったキャンペーン効果検証V1
傾向スコアを使ったキャンペーン効果検証V1傾向スコアを使ったキャンペーン効果検証V1
傾向スコアを使ったキャンペーン効果検証V1Kazuya Obanayama
 
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた. .
 
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics
 
心理学におけるベイズ統計の流行を整理する
心理学におけるベイズ統計の流行を整理する心理学におけるベイズ統計の流行を整理する
心理学におけるベイズ統計の流行を整理するHiroshi Shimizu
 
The Power of a Proper North Star Metric
The Power of a Proper North Star MetricThe Power of a Proper North Star Metric
The Power of a Proper North Star MetricRocketSource
 
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan Hamed
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan HamedUplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan Hamed
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan HamedRising Media Ltd.
 

What's hot (20)

B2B SaaS north star metric 2019 2020
B2B SaaS north star metric 2019 2020B2B SaaS north star metric 2019 2020
B2B SaaS north star metric 2019 2020
 
Lean Analytics Cycle
Lean Analytics CycleLean Analytics Cycle
Lean Analytics Cycle
 
Slides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclassSlides from Growthcon 2014 Lean Analytics masterclass
Slides from Growthcon 2014 Lean Analytics masterclass
 
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...
The Playbook to Running Growth Experiments at Scale with Ex Machina Founder G...
 
Basics of AB testing in online products
Basics of AB testing in online productsBasics of AB testing in online products
Basics of AB testing in online products
 
What's a Growth Stack? And why you should build one.
What's a Growth Stack? And why you should build one. What's a Growth Stack? And why you should build one.
What's a Growth Stack? And why you should build one.
 
Rick Rasmussen - Product market fit
Rick Rasmussen - Product market fitRick Rasmussen - Product market fit
Rick Rasmussen - Product market fit
 
Google analytics 4 - présentation détaillée
Google analytics 4 - présentation détailléeGoogle analytics 4 - présentation détaillée
Google analytics 4 - présentation détaillée
 
Growth Hacking Workshop: AAU
Growth Hacking Workshop: AAUGrowth Hacking Workshop: AAU
Growth Hacking Workshop: AAU
 
The Power of A/B Testing
The Power of A/B TestingThe Power of A/B Testing
The Power of A/B Testing
 
Causality without headaches
Causality without headachesCausality without headaches
Causality without headaches
 
Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)
 
傾向スコアを使ったキャンペーン効果検証V1
傾向スコアを使ったキャンペーン効果検証V1傾向スコアを使ったキャンペーン効果検証V1
傾向スコアを使ったキャンペーン効果検証V1
 
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた
『予測にいかす統計モデリングの基本』の売上データの分析をトレースしてみた
 
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)
 
心理学におけるベイズ統計の流行を整理する
心理学におけるベイズ統計の流行を整理する心理学におけるベイズ統計の流行を整理する
心理学におけるベイズ統計の流行を整理する
 
The Power of a Proper North Star Metric
The Power of a Proper North Star MetricThe Power of a Proper North Star Metric
The Power of a Proper North Star Metric
 
A/B testing
A/B testingA/B testing
A/B testing
 
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan Hamed
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan HamedUplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan Hamed
Uplift Modelling as a Tool for Making Causal Inferences at Shopify - Mojan Hamed
 
RFM Segmentation
RFM SegmentationRFM Segmentation
RFM Segmentation
 

Similar to Lean Analytics workshop for Dublin City University, April 2014

online startups metrics
online startups metrics online startups metrics
online startups metrics Hatem Kameli
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Lean Startup Co.
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsMeasureWorks
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Kate O'Neill
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking jeroentjepkema
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
Making money with android apps
Making money with android appsMaking money with android apps
Making money with android appsNathan Mellor
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungMattan Griffel
 
Introduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFLean Analytics
 
Big data can be used at SME's too
Big data can be used at SME's tooBig data can be used at SME's too
Big data can be used at SME's tooGeorge Antony
 
What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)Dave LaFontaine
 
101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon ParisDvir Reznik
 
Lean Analytics & Analytics Dashboards
Lean Analytics & Analytics DashboardsLean Analytics & Analytics Dashboards
Lean Analytics & Analytics DashboardsYves Ferket
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analyticsKartik Narayanan
 
Everything you ever wanted to know about Google Analytics, but were afraid to...
Everything you ever wanted to know about Google Analytics, but were afraid to...Everything you ever wanted to know about Google Analytics, but were afraid to...
Everything you ever wanted to know about Google Analytics, but were afraid to...SoCal UX Camp
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfjeroen_tjepkema
 

Similar to Lean Analytics workshop for Dublin City University, April 2014 (20)

online startups metrics
online startups metrics online startups metrics
online startups metrics
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
 
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
Startup Metrics: The Data That Will Make or Break Your Business by Alistair C...
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
SBC Growth Week - Lean Analytics
SBC Growth Week - Lean AnalyticsSBC Growth Week - Lean Analytics
SBC Growth Week - Lean Analytics
 
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
Analytics, Search, Social Media, and Optimization: Why Has Marketing Gotten S...
 
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
Adobe User Group Amsterdam - Correlation between Innovation & Growth Hacking
 
Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
Making money with android apps
Making money with android appsMaking money with android apps
Making money with android apps
 
Growth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-YoungGrowth Hacking with Cassie Lancellotti-Young
Growth Hacking with Cassie Lancellotti-Young
 
Introduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SFIntroduction to Lean Analytics for Lean Startup Circle SF
Introduction to Lean Analytics for Lean Startup Circle SF
 
Big data can be used at SME's too
Big data can be used at SME's tooBig data can be used at SME's too
Big data can be used at SME's too
 
What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)What is a Creative Date Scientist (and why the $@%! do we need one?)
What is a Creative Date Scientist (and why the $@%! do we need one?)
 
101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris101 of Mobile Marketing - Droidcon Paris
101 of Mobile Marketing - Droidcon Paris
 
Lean Analytics & Analytics Dashboards
Lean Analytics & Analytics DashboardsLean Analytics & Analytics Dashboards
Lean Analytics & Analytics Dashboards
 
Introduction to lean analytics
Introduction to lean analyticsIntroduction to lean analytics
Introduction to lean analytics
 
Everything you ever wanted to know about Google Analytics, but were afraid to...
Everything you ever wanted to know about Google Analytics, but were afraid to...Everything you ever wanted to know about Google Analytics, but were afraid to...
Everything you ever wanted to know about Google Analytics, but were afraid to...
 
Building data science teams
Building data science teamsBuilding data science teams
Building data science teams
 
LTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdfLTK - FC - Supply Chain - Startup Challenge v3.pdf
LTK - FC - Supply Chain - Startup Challenge v3.pdf
 

More from Lean Analytics

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsLean Analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Lean Analytics
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014Lean Analytics
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean Analytics
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics
 
OnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsLean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)Lean Analytics
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Lean Analytics
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Lean Analytics
 
7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can HelpLean Analytics
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine MontrealLean Analytics
 

More from Lean Analytics (13)

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
 
OnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean AnalyticsOnLab Japan introduction to Lean Analytics
OnLab Japan introduction to Lean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)Understanding Lean Analytics (and how analytics helps businesses win)
Understanding Lean Analytics (and how analytics helps businesses win)
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
 
7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine Montreal
 

Recently uploaded

5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 

Recently uploaded (17)

5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 

Lean Analytics workshop for Dublin City University, April 2014

  • 3. Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  • 4. The core of Lean is iteration.
  • 5. Most startups don’t know what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
  • 7. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  • 9. Analytics is the measurement of movement towards your business goals.
  • 10. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 11. I have two kids. At least one of them is a girl.
  • 12. What are the chances the other is a boy?
  • 14. 2 of 3 (66%) are boys. GB GG BG
  • 16. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.
  • 17. The simplest rule bad metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  • 18. Metrics help you know yourself. Acquisition Hybrid Loyalty 70% of retailers 20% of retailers 10% of retailers You are just like Customers that buy >1x in 90d Once 2-2.5 per year >2.5 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  • 19. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 20. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 21. Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t know we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 22. MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 23. Which of these two companies is doing better?
  • 24.   January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January February March April May $5 $3 $2 $1 $0.5 $6 $4 $2 $1 $7 $6 $5   $8 $7       $9 How about this one?
  • 25. Cohort 1 2 3 4 5 January February March April May Averages $5 $3 $2 $1 $0.5 $6 $4 $2 $1   $7 $6 $5     $8 $7       $9         $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 26. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 27. A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 28. Which means it’s time to talk about correlation.
  • 29. 1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  • 30. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 31. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 33. Is social action a leading indicator of donation? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
  • 35. Why is Nigerian spam so badly written?
  • 36. Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  • 37. Turns out the word “Nigeria” is the best way to identify promising prospects.
  • 38. Nigerian spammers really understand their target market. They see past vanity metrics.
  • 39. The Lean Analytics framework.
  • 40. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 41. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 42. Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  • 43. Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
  • 44. Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient reads survey question Recipient responds to question Recipient installs mobile app Recipient creates account, profile Recipient sees survey results Recipient can edit profile, etc. 10-25%RESPONSERATE 70-90%RESPONSERATE
  • 45. Six business model archetypes (Yours is probably a blend of these.)
  • 46. E-commerce SaaS (freemium?) Mobile app (gaming) Two sided marketplace Media User generated content
  • 47. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  • 48. Model + Stage = One Metric That Matters. One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  • 51. In a startup, focus is hard to achieve.
  • 52. Having only one metric addresses this problem.
  • 54. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 55. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 56. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile app User-gen content 2-sided market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  • 58. What other metrics do you want to know about?
  • 59. Drawing some lines in the sand.
  • 60. A company loses a quarter of its customers every year. Is this good or bad?
  • 61. Not knowing what normal is makes you do stupid things.
  • 62. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 63. Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  • 64. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 65. Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  • 66. Baseline: Calculating customer lifetime 25% monthly churn 100/25=4 The average customer lasts 4 months 5% monthly churn 100/5=20 The average customer lasts 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
  • 67. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  • 69. Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 70. Do AirBnB hosts get more business if their property is professionally photographed?
  • 71. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 72. 5,000 shoots per month by February 2012
  • 73. Hang on a second.
  • 74. Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
  • 75. Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  • 76. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data: find a commonality Without data: make a good guess
  • 77. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  • 79. I lied. Everyone is a tech company.
  • 81. Let’s pick on restaurants for a while.
  • 82. A line in the sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
  • 83. A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50 reservations at 5PM 250 covers that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
  • 85. Is tip amount a leading indicator of long- term revenue?
  • 86. Why does every table get the same menu?
  • 87. Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf
  • 89. Growth hacking (is a word you should hate but will hear a lot about.)
  • 92. A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (from the 2012 Growth Hacking conference) (These are also great segments to analyze.) The leading indicator
  • 93. • Growth hacking is simply what marketing should have been doing, but it fell in love with Don Draper and opinions along the way • Optimize a factor you think is correlated with growth The growth hack
  • 98. What if you’re in a big organization?
  • 99. The Zero Overhead principle A central theme to this new wave of innovation is the application of core product tenets from the consumer space to the enterprise. In particular, a universal lesson that I keep sharing with all entrepreneurs building for the enterprise is the Zero Overhead Principle: no feature may add training costs to the user. DJ Patil
  • 100. The B2B stereotype http://www.techdigest.tv/2007/02/im_a_pc_im_a_ma.html Domain expert Disruption expert •Domain expert knows industry and the problem domain. Has a Rolodex; proxy for customers. •Disruption expert knows tech that will produce a change Sees beyond the current model.
  • 101. The Lean Analytics lifecycle of an Intrapreneur Empathy Consulting to test ideas and bootstrap the business Lock-in, IP control, overfitting Stickiness Standardization and integration; shift from custom to generic Ability to integrate; support Virality Word of mouth, references, case studies Bad vibes; exclusivity Revenue Growing direct sales, professional services, support Pipeline, revenue recognition, comp Scale Channels, analysts, ecosystems, APIs, vertically targeted products Crossing the chasm; Gorillas Stage Do this Fear this
  • 102. Enterprise-focused startups: Metrics that matter • Ease of customer engagement and feedback • Alpha/beta pipeline • Stickiness and usability • Integration costs • User engagement with the app • Disentanglement after a sale • Support costs • User groups and feedback • Pitch success for channel tools • Barriers to exit
  • 103. What if you’re in a big organization?
  • 104. If a startup is an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.
  • 105. The Lean Analytics lifecycle of an Intrapreneur Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby Beforehand Get buy-in Political fallout
  • 107. Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  • 109. Use outliers and missed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
  • 110. Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  • 111. Focus on the desired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
  • 112. Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
  • 113. Know what has to be built in-house SAP integration Employment law
  • 114. Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
  • 115. When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
  • 116. Everything’s an excuse to experiment
  • 117. Find other ways to collect data; everything is an experiment.
  • 118. Don’t just collect data, chase it.
  • 119. Some tools and traps
  • 120. Traction graphs Your business model The stage you’re at Your one metric ... change often if you’re doing it right. So how do you track that over time?
  • 121. Traction graphs Jan Feb Mar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient This axis changes for each metric
  • 122. Traction graphs Jan Feb Mar Apr May Jun Signups per day Conversion rate Churn rate Viral coefficient 0%
  • 123. Use vanity to get to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one x Web traffic Revenue Activation Cart Size Conversion rate
  • 124. The three threes Three assumptions What big bets are you making? •“People will answer questions” •“Organizers are frustrated with how to run conferences” •“We'll make money from parents” •“Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) •Product enhancements •Marketing strategies Three experiments to run •Feature tests •Continuous deployment •A/B testing •Customer survey
  • 125. The three threes Three assumptions Three actions to take Three experiments to run Monthly Weekly Daily Board, investors, founders Executive team Employees Strategy Tactics Execution
  • 126. The three threes Three assumptions Three actions to take Three experiments to run Get more people Increase answer % Test better questions Change the UI Test timings Questions from peers Many people will answer questions
  • 127. The problem-solution canvas CURRENT STATUS • List key metrics you’re tracking, where they’re at, and compare with last few weeks • How are things trending? LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? • What was accomplished? • On track: YES / NO? The Goal is to Learn
  • 128. The problem-solution canvas HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS Problem #1 (put name here) • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Define goals for the metric HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) Problem #2 (put name here)
  • 129. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 130. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 132. Once, a leader convinced others in the absence of data.
  • 133. Now, a leader knows what questions to ask.
  • 135. Follow-on: The other business model system diagrams
  • 138. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!