SlideShare a Scribd company logo
1 of 62
www.leananalyticsbook.com
@leananalytics
@byosko | @acroll
Lean Analytics
Use data to build a
better business faster.
A (really) quick
intro to Lean principles
The basic Lean message is
learn and adapt, fast.
Kevin Costner is a lousy entrepreneur.
Don’t sell what you can make.
Make what you can sell.
In a startup, the purpose
of analytics is to iterate to
a product/market fit before
the money runs out.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where things
fall apart.
No data, no
learning.
The five Stages of Lean Analytics
Empathy
Stickiness
Virality
Revenue
Scale
Thestageyou’reat
E-commerce SaaS Media
Mobile
app
User-gen
content
2-sided
market
The business you’re in
One Metric
That Matters.
If it won’t change
how you behave,
it’s a
bad
metric.
http://www.flickr.com/photos/circasassy/7858155676/
Five lessons beyond the basics
• Mining your existing data
• A new product every day
• Are you at your local maximum?
• It’s different when you’re big
• Build analytics into how you operate
Mining your existing data
Moving beyond guesswork.
Identify a key business problem,
pick the OMTM, draw a line in the
sand, and get started.
Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
The Lean Analytics Cycle
Do hosts withDo hosts with
professionalprofessional
photography getphotography get
more business?more business?
Do hosts withDo hosts with
professionalprofessional
photography getphotography get
more business?more business?
Airbnb experiments...
Professional photography helps Airbnb’s business
Gut instinct
Concierge MVP
20 photographers in the field
Test results
Two to three times more bookings!
Back to the beginning
Use additional data to keep experimenting
5,000 shoots/month in Feb. 2012
Hang on a second...
Really?
Professional photography helps Airbnb’s business
Gut instinct
Draw a new line
Pivot or
give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in production
Design a test
Hypothesis
With data:
find a
commonality
Without data:
make a good
guess
Find a potential
improvement
Draw a linePick a KPI
Remember this?
“Gee, those houses
that do well look
really nice.”
Maybe it’s the
camera.
“Computer: What do
all the highly rented
houses have in
common?”
Camera model.
With data:
find a
commonality
Without data:
make a good
guess
The only thing worse than bad
feedback is no feedback at all.
- Dave McClure, Startupfest 2012
Because it gives
you nothing to analyze.
Virality stage:
Circle of Moms finds an
engaged market
• Stage: Stickiness
• Model: UGC
• Launched as Circle of Friends in 2007, it was a way for small groups to
interact atop Facebook’s platform; but when engagement wasn’t good
enough, the founders decided to dig deeper.
The problem: Not enough engagement
• Too few people were actually using the product
• Less than 20% of any circles had any activity after their initial creation
• A few million monthly uniques from 10M registered users, but no sustained
traction
A new product every day
Why cohorts matter more than you think.
Segments, cohorts, A/B, and multivariates
Segment:
Cross-sectional
comparison of all
people divided by
some attribute
(age, gender, etc.)
☀
☁
Cohort:
Comparison of
similar groups
along a timeline.
A/B test:
Changing one
thing (i.e. color)
and measuring
the result (i.e.
revenue.)
Multivariate
analysis:
Changing several
things at once to see
which correlates with
a result.
☀
☁
☀
☁
JanJan
FebFeb
MarMar
AprApr
MayMay
Why use cohorts? Here’s an example.
January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this
company
growing or
stagnating?
Cohort 1 2 3 4 5
How about
this one?
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
Two views of the same company
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1
March $7 $6 $5
April $8 $7
May $9
Averages $7 $5 $3 $1 $0.5
Look at the
same data
in cohorts
Are we at a local maximum?
You need optimization and inspiration.
The problem with local maxima
(http://www.slideshare.net/bokardo/metricsdriven-design-4317168)
The vertebrate eye is “backwards
and upside down”. Light travels
through cornea, lens, aqueous
fluid, blood vessels, ganglion
cells, amacrine cells, horizontal
cells, and bipolar cells.
The cephalopod eye is
constructed the “right way
out”, with the nerves attached
to the rear of the retina. No
blind spot.
Extension of the brain Invagination of the head
It’s different when you’re big
Tilting at the corporate windmill.
“A startup is an organization
designed to search for a
sustainable, repeatable
business model.”
- Steve Blank
Remember when I said,
“A big company is an
organization designed to
perpetuate a business model.”
- Me
In other words, if your job is
change you have your
work cut out for you.
Span of control and the railroads
The story of Daniel C. McCallum
http://www.flickr.com/photos/art_es_anna/288880795/
Barnes and Noble tried pretty hard.
• $1B invested in
Nook
• $475M operating
loss in April
• CEO gone
The Skunk Works
Since large organizations
make data that
perpetuates things, you
have to willfully ignore
common wisdom at the
outset.
http://www.flickr.com/photos/jb912/8940173789
Disruption may not fit neatly into
an existing part of the
organization, so you need to
incubate it internally for a while.
Unlike a VC or startup,
when the initiative fails the
organization still learns.
Big organizations might not
innovate at speed—but they
innovate at scale.
The Lean Analytics lifecycle for an Intrapreneur
Empathy
Find problems; don’t test demand.
Skip the business case, do
analytics
Entitled, aggrieved
customers
Stickiness
Know your real minimum based on
expectations, regulations
Hidden “must haves”,
feature creep
Virality
Build inherent virality in from the
start; attention is the new currency
Luddites who don’t
understand sharing
Revenue
Consider the ecosystem, channels,
and established agreements
Channel conflict,
resistance, contracts
Scale
Hand the baton to others gracefully Hating what happens
to your baby
Stage Do this Fear this
Beforehand
Get buy-in Political fallout
A large organization needs a
portfolio of investments,
judged by different criteria.
Treasury
bonds
Venture
capital
A large organization needs a
portfolio of investments,
judged by different criteria.
Safe
Predictable
Evolutionary
Achieve the
business plan
Risky
Uncertain
Revolutionary
Discover the
business model
Times of
peace
Times of
war
Acquisition or disruption?
Innovate fast:
nothing to lose
Innovate slowly: conform to brand,
legislation, approvals
Found
P/M fit
Found
P/M fit
Integration complexity,
founder churn, indigestion
Biz
case
Scale
Scale
Biz
model
VC
pitch
Moving from box to box
• How businesses think
about products or
companies
• Lean is about moving
up and to the right
Frame the project as a
learning exercise; product
creation is almost accidental.
Learn it’s
a failure
Learn it’s
a success
Industry
knowledge
Industry
knowledge
Possible
product
Business plan
metrics here
Market
Business
model
metrics here
Lean
product
Lines in the
sand here
Build analytics into how you
operate the business
Three threes and the Problem-Solution Canvas
Three threes
Three
assumptions
What big bets are you making?
•“People will answer questions”
•“Organizers are frustrated with how to run conferences”
•“We’ll make money from parents”
•“Amazon is reliable enough for our users.”
Three actions
to take
What are you doing to make these assumptions happen
(or identify they’re wrong and change course?)
•Product enhancements
•Marketing strategies
Three experiments
to run
What are you going to learn today?
•Feature tests
•Continuous deployment
•A/B testing
•Customer survey
Three threes
Monthly
Weekly
Daily
Board, investors,
founders
Executive team
Employees
Strategy
Tactics
Execution
Three
assumptions
Three actions
to take
Three experiments
to run
Three threes
Get more
people
Increase
answer %
Test better
questions
Change
the UI
Test
timings
Questions
from peers
Many people will
answer questions
Three
assumptions
Three actions
to take
Three experiments
to run
In the end, be subversive.
http://www.flickr.com/photos/bootbearwdc/1243690099/
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko

More Related Content

What's hot

Growth hacking secrets
Growth hacking secrets Growth hacking secrets
Growth hacking secrets alig12
 
Lean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Startup Co.
 
Metrics for early stage startups
Metrics for early stage startupsMetrics for early stage startups
Metrics for early stage startupsAndreas Klinger
 
21 Actionable Growth Hacking Tactics
21 Actionable Growth Hacking Tactics21 Actionable Growth Hacking Tactics
21 Actionable Growth Hacking TacticsJon Yongfook
 
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics
 
The Science behind Viral marketing
The Science behind Viral marketingThe Science behind Viral marketing
The Science behind Viral marketingDavid Skok
 
Building the Billion Dollar SaaS Unicorn: CEO Guide
Building the Billion Dollar SaaS Unicorn: CEO GuideBuilding the Billion Dollar SaaS Unicorn: CEO Guide
Building the Billion Dollar SaaS Unicorn: CEO GuideKelly Schwedland
 
Optimize Your Funnel By Getting Inside Your Buyer's Head
Optimize Your Funnel By Getting Inside Your Buyer's HeadOptimize Your Funnel By Getting Inside Your Buyer's Head
Optimize Your Funnel By Getting Inside Your Buyer's HeadDavid Skok
 
Growth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processGrowth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processRuben Hamilius
 
Outbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowOutbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowDavid Skok
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash CourseDavid Fallarme
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth HackerTim Homuth
 
Getting to Product Market Fit - An Overview of Customer Discovery & Validation
Getting to Product Market Fit - An Overview of Customer Discovery & ValidationGetting to Product Market Fit - An Overview of Customer Discovery & Validation
Getting to Product Market Fit - An Overview of Customer Discovery & ValidationJason Evanish
 
Product Analytics 101 by Pendo VP of Products
Product Analytics 101 by Pendo VP of ProductsProduct Analytics 101 by Pendo VP of Products
Product Analytics 101 by Pendo VP of ProductsProduct School
 
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptx
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptxThe Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptx
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptxEuropean Innovation Academy
 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS MetricsChristoph Janz
 
Ultimate Guide to Funnel Optimization
Ultimate Guide to Funnel OptimizationUltimate Guide to Funnel Optimization
Ultimate Guide to Funnel OptimizationSean Johnson
 
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdfEuropean Innovation Academy
 
Email Intensive 2022.pdf
Email Intensive 2022.pdfEmail Intensive 2022.pdf
Email Intensive 2022.pdfPerry Belcher
 

What's hot (20)

Growth hacking secrets
Growth hacking secrets Growth hacking secrets
Growth hacking secrets
 
Lean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business FasterLean Analytics: Using Data to Build a Better Business Faster
Lean Analytics: Using Data to Build a Better Business Faster
 
Metrics for early stage startups
Metrics for early stage startupsMetrics for early stage startups
Metrics for early stage startups
 
21 Actionable Growth Hacking Tactics
21 Actionable Growth Hacking Tactics21 Actionable Growth Hacking Tactics
21 Actionable Growth Hacking Tactics
 
Zero to 50m
Zero to 50m Zero to 50m
Zero to 50m
 
Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)Lean Analytics workshop (from Lean Startup Conf)
Lean Analytics workshop (from Lean Startup Conf)
 
The Science behind Viral marketing
The Science behind Viral marketingThe Science behind Viral marketing
The Science behind Viral marketing
 
Building the Billion Dollar SaaS Unicorn: CEO Guide
Building the Billion Dollar SaaS Unicorn: CEO GuideBuilding the Billion Dollar SaaS Unicorn: CEO Guide
Building the Billion Dollar SaaS Unicorn: CEO Guide
 
Optimize Your Funnel By Getting Inside Your Buyer's Head
Optimize Your Funnel By Getting Inside Your Buyer's HeadOptimize Your Funnel By Getting Inside Your Buyer's Head
Optimize Your Funnel By Getting Inside Your Buyer's Head
 
Growth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about processGrowth Hacking / Marketing 101: It's about process
Growth Hacking / Marketing 101: It's about process
 
Outbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flowOutbound prospecting for highly targeted lead flow
Outbound prospecting for highly targeted lead flow
 
Growth Hacking: A Crash Course
Growth Hacking: A Crash CourseGrowth Hacking: A Crash Course
Growth Hacking: A Crash Course
 
Think Like A Growth Hacker
Think Like A Growth HackerThink Like A Growth Hacker
Think Like A Growth Hacker
 
Getting to Product Market Fit - An Overview of Customer Discovery & Validation
Getting to Product Market Fit - An Overview of Customer Discovery & ValidationGetting to Product Market Fit - An Overview of Customer Discovery & Validation
Getting to Product Market Fit - An Overview of Customer Discovery & Validation
 
Product Analytics 101 by Pendo VP of Products
Product Analytics 101 by Pendo VP of ProductsProduct Analytics 101 by Pendo VP of Products
Product Analytics 101 by Pendo VP of Products
 
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptx
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptxThe Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptx
The Science and Emotion of Neuromarketing_Nina Stanley_EIA Porto 2022.pptx
 
9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics9 Worst Practices in SaaS Metrics
9 Worst Practices in SaaS Metrics
 
Ultimate Guide to Funnel Optimization
Ultimate Guide to Funnel OptimizationUltimate Guide to Funnel Optimization
Ultimate Guide to Funnel Optimization
 
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf
26_07_Marketing 101_Maeghan Smulders_EIA Porto 2022.pdf
 
Email Intensive 2022.pdf
Email Intensive 2022.pdfEmail Intensive 2022.pdf
Email Intensive 2022.pdf
 

Viewers also liked

Quantifying the Invisible Audience in Social Networks
Quantifying the Invisible Audience in Social NetworksQuantifying the Invisible Audience in Social Networks
Quantifying the Invisible Audience in Social NetworksMichael Bernstein
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean Analytics
 
Phil Williford Lean Enterprise
Phil Williford Lean EnterprisePhil Williford Lean Enterprise
Phil Williford Lean Enterprisepmwilliford
 
Starting a new game business
Starting a new game businessStarting a new game business
Starting a new game businessJussi Laakkonen
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014Lean Analytics
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics
 
Korean mobile game marketing trend and strategy
Korean mobile game marketing trend and strategyKorean mobile game marketing trend and strategy
Korean mobile game marketing trend and strategyKyung-a Park
 
The Future is Operations: Why Mobile Games Need Backends
The Future is Operations: Why Mobile Games Need BackendsThe Future is Operations: Why Mobile Games Need Backends
The Future is Operations: Why Mobile Games Need BackendsJames Gwertzman
 
Launching a Free-to-Play Game: Strategies, Risks, and Pitfalls
Launching a Free-to-Play Game: Strategies, Risks, and PitfallsLaunching a Free-to-Play Game: Strategies, Risks, and Pitfalls
Launching a Free-to-Play Game: Strategies, Risks, and PitfallsEric Seufert
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
Mobile Game Go-to-Market case study
Mobile Game Go-to-Market case studyMobile Game Go-to-Market case study
Mobile Game Go-to-Market case studyDon Lim
 

Viewers also liked (14)

Quantifying the Invisible Audience in Social Networks
Quantifying the Invisible Audience in Social NetworksQuantifying the Invisible Audience in Social Networks
Quantifying the Invisible Audience in Social Networks
 
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)
 
Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014Lean analytics from Web A Quebec mars 2014
Lean analytics from Web A Quebec mars 2014
 
Phil Williford Lean Enterprise
Phil Williford Lean EnterprisePhil Williford Lean Enterprise
Phil Williford Lean Enterprise
 
Starting a new game business
Starting a new game businessStarting a new game business
Starting a new game business
 
training presentation
training presentationtraining presentation
training presentation
 
Croll lean analytics workshop (3h) - lean ux nyc april 2014
Croll   lean analytics workshop (3h) - lean ux nyc april 2014Croll   lean analytics workshop (3h) - lean ux nyc april 2014
Croll lean analytics workshop (3h) - lean ux nyc april 2014
 
Lean Analytics for Nikkei BP
Lean Analytics for Nikkei BPLean Analytics for Nikkei BP
Lean Analytics for Nikkei BP
 
Spark Lean Intrapreneur Model
Spark Lean Intrapreneur ModelSpark Lean Intrapreneur Model
Spark Lean Intrapreneur Model
 
Korean mobile game marketing trend and strategy
Korean mobile game marketing trend and strategyKorean mobile game marketing trend and strategy
Korean mobile game marketing trend and strategy
 
The Future is Operations: Why Mobile Games Need Backends
The Future is Operations: Why Mobile Games Need BackendsThe Future is Operations: Why Mobile Games Need Backends
The Future is Operations: Why Mobile Games Need Backends
 
Launching a Free-to-Play Game: Strategies, Risks, and Pitfalls
Launching a Free-to-Play Game: Strategies, Risks, and PitfallsLaunching a Free-to-Play Game: Strategies, Risks, and Pitfalls
Launching a Free-to-Play Game: Strategies, Risks, and Pitfalls
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
Mobile Game Go-to-Market case study
Mobile Game Go-to-Market case studyMobile Game Go-to-Market case study
Mobile Game Go-to-Market case study
 

Similar to Lean analytics: Five lessons beyond the basics

Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018Lean Startup Circle Montreal
 
How To Fail: 25 Secrets Learned through Failure
How To Fail: 25 Secrets Learned through FailureHow To Fail: 25 Secrets Learned through Failure
How To Fail: 25 Secrets Learned through FailureTaylor Davidson
 
Final cycles overview jan 2019 with toolkit
Final cycles overview jan 2019 with toolkitFinal cycles overview jan 2019 with toolkit
Final cycles overview jan 2019 with toolkitBryan Cassady
 
A new leadership framework
A new leadership frameworkA new leadership framework
A new leadership frameworkFarooq Omar
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Startup Co.
 
Fundraising Strategy - useful tools that really work IoF 2011
Fundraising Strategy - useful tools that really work IoF 2011Fundraising Strategy - useful tools that really work IoF 2011
Fundraising Strategy - useful tools that really work IoF 2011Simon Burne
 
QBO WORQshop: Are you ready to fundraise?
QBO WORQshop: Are you ready to fundraise?QBO WORQshop: Are you ready to fundraise?
QBO WORQshop: Are you ready to fundraise?Joseph de Leon
 
ENTREPRENEURSHIP_PRESENTATION[1].pdf
ENTREPRENEURSHIP_PRESENTATION[1].pdfENTREPRENEURSHIP_PRESENTATION[1].pdf
ENTREPRENEURSHIP_PRESENTATION[1].pdfMUHUMUZAONAN1
 
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)Achieving Business Excellence (a ChangeThis Manifesto by John Spence)
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)Samuli Pahkala
 
2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxswEric Ries
 
EDHEC Course Introduction
EDHEC Course Introduction EDHEC Course Introduction
EDHEC Course Introduction Bryan Cassady
 
Master the essentials of conversion optimization
Master the essentials of conversion optimizationMaster the essentials of conversion optimization
Master the essentials of conversion optimizationArnas Rackauskas
 
Using Powerful Questions to Drive Your Information Strategy
Using Powerful Questions to Drive Your Information StrategyUsing Powerful Questions to Drive Your Information Strategy
Using Powerful Questions to Drive Your Information StrategyRob Saker
 
How to make your corporate management think and act like a startup. (by @boar...
How to make your corporate management think and act like a startup. (by @boar...How to make your corporate management think and act like a startup. (by @boar...
How to make your corporate management think and act like a startup. (by @boar...Board of Innovation
 
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018 Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018 Bartek Janowicz
 
Info Product Creation Manifesto
Info Product Creation ManifestoInfo Product Creation Manifesto
Info Product Creation ManifestoGreg Rollett
 
InnoVits - Aiesec 29 ottobre 2014
InnoVits - Aiesec 29 ottobre 2014InnoVits - Aiesec 29 ottobre 2014
InnoVits - Aiesec 29 ottobre 2014InnoVits
 

Similar to Lean analytics: Five lessons beyond the basics (20)

Alistaire croll lean analytics - montreal lean startup circle - september 2018
Alistaire croll   lean analytics - montreal lean startup circle - september 2018Alistaire croll   lean analytics - montreal lean startup circle - september 2018
Alistaire croll lean analytics - montreal lean startup circle - september 2018
 
How To Fail: 25 Secrets Learned through Failure
How To Fail: 25 Secrets Learned through FailureHow To Fail: 25 Secrets Learned through Failure
How To Fail: 25 Secrets Learned through Failure
 
Final cycles overview jan 2019 with toolkit
Final cycles overview jan 2019 with toolkitFinal cycles overview jan 2019 with toolkit
Final cycles overview jan 2019 with toolkit
 
A new leadership framework
A new leadership frameworkA new leadership framework
A new leadership framework
 
Gravity v0.2
Gravity v0.2Gravity v0.2
Gravity v0.2
 
Lean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair CrollLean Analytics for Intrapreneurs by Allistair Croll
Lean Analytics for Intrapreneurs by Allistair Croll
 
Fundraising Strategy - useful tools that really work IoF 2011
Fundraising Strategy - useful tools that really work IoF 2011Fundraising Strategy - useful tools that really work IoF 2011
Fundraising Strategy - useful tools that really work IoF 2011
 
QBO WORQshop: Are you ready to fundraise?
QBO WORQshop: Are you ready to fundraise?QBO WORQshop: Are you ready to fundraise?
QBO WORQshop: Are you ready to fundraise?
 
ENTREPRENEURSHIP_PRESENTATION[1].pdf
ENTREPRENEURSHIP_PRESENTATION[1].pdfENTREPRENEURSHIP_PRESENTATION[1].pdf
ENTREPRENEURSHIP_PRESENTATION[1].pdf
 
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)Achieving Business Excellence (a ChangeThis Manifesto by John Spence)
Achieving Business Excellence (a ChangeThis Manifesto by John Spence)
 
2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw2011 03 11 eric ries - the lean startup sxsw
2011 03 11 eric ries - the lean startup sxsw
 
EDHEC Course Introduction
EDHEC Course Introduction EDHEC Course Introduction
EDHEC Course Introduction
 
Master the essentials of conversion optimization
Master the essentials of conversion optimizationMaster the essentials of conversion optimization
Master the essentials of conversion optimization
 
Using Powerful Questions to Drive Your Information Strategy
Using Powerful Questions to Drive Your Information StrategyUsing Powerful Questions to Drive Your Information Strategy
Using Powerful Questions to Drive Your Information Strategy
 
How to make your corporate management think and act like a startup. (by @boar...
How to make your corporate management think and act like a startup. (by @boar...How to make your corporate management think and act like a startup. (by @boar...
How to make your corporate management think and act like a startup. (by @boar...
 
Genesis Startup Master Class
Genesis Startup Master ClassGenesis Startup Master Class
Genesis Startup Master Class
 
Genesis
GenesisGenesis
Genesis
 
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018 Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018
Otto Freijser - Perpetulon - Lean Startup Night Warsaw - Feb 13th, 2018
 
Info Product Creation Manifesto
Info Product Creation ManifestoInfo Product Creation Manifesto
Info Product Creation Manifesto
 
InnoVits - Aiesec 29 ottobre 2014
InnoVits - Aiesec 29 ottobre 2014InnoVits - Aiesec 29 ottobre 2014
InnoVits - Aiesec 29 ottobre 2014
 

More from Lean Analytics

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsLean Analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Lean Analytics
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Lean Analytics
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)Lean Analytics
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Lean Analytics
 
7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can HelpLean Analytics
 
Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Lean Analytics
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine MontrealLean Analytics
 

More from Lean Analytics (10)

Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analyticsMelbourne Business School - mba talk october 14 - croll - 40m - lean analytics
Melbourne Business School - mba talk october 14 - croll - 40m - lean analytics
 
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
 
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
Slides for the day-long Lean Analytics workshop at the 2014 Lean Startup conf...
 
Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014Lean Analytics workshop for Dublin City University, April 2014
Lean Analytics workshop for Dublin City University, April 2014
 
Lean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for AmericaLean Analytics and Local Government - Alistair Croll - Code for America
Lean Analytics and Local Government - Alistair Croll - Code for America
 
Big data tokyo (extended version)
Big data tokyo  (extended version)Big data tokyo  (extended version)
Big data tokyo (extended version)
 
Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)Making Sense of the Numbers (Lean Analytics)
Making Sense of the Numbers (Lean Analytics)
 
7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help7 Myths of Lean and How Analytics Can Help
7 Myths of Lean and How Analytics Can Help
 
Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)Introduction to Lean Analytics webinar (O'Reilly)
Introduction to Lean Analytics webinar (O'Reilly)
 
Lean Startup Machine Montreal
Lean Startup Machine MontrealLean Startup Machine Montreal
Lean Startup Machine Montreal
 

Recently uploaded

WSMM Technology February.March Newsletter_vF.pdf
WSMM Technology February.March Newsletter_vF.pdfWSMM Technology February.March Newsletter_vF.pdf
WSMM Technology February.March Newsletter_vF.pdfJamesConcepcion7
 
5 Digital Onboarding Trends To Watch In 2024.pdf
5 Digital Onboarding Trends To Watch In 2024.pdf5 Digital Onboarding Trends To Watch In 2024.pdf
5 Digital Onboarding Trends To Watch In 2024.pdfVeri5digital
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdfShaun Heinrichs
 
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdftrending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdfMintel Group
 
Customizable Contents Restoration Training
Customizable Contents Restoration TrainingCustomizable Contents Restoration Training
Customizable Contents Restoration TrainingCalvinarnold843
 
Welding Electrode Making Machine By Deccan Dynamics
Welding Electrode Making Machine By Deccan DynamicsWelding Electrode Making Machine By Deccan Dynamics
Welding Electrode Making Machine By Deccan DynamicsIndiaMART InterMESH Limited
 
Pitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckPitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckHajeJanKamps
 
Certificate of Good Standing Ferber Enterprises USA LLC
Certificate of Good Standing Ferber Enterprises USA LLCCertificate of Good Standing Ferber Enterprises USA LLC
Certificate of Good Standing Ferber Enterprises USA LLCFerber Enterprises
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers referencessuser2c065e
 
Neha Jhalani Hiranandani: A Guide to Her Life and Career
Neha Jhalani Hiranandani: A Guide to Her Life and CareerNeha Jhalani Hiranandani: A Guide to Her Life and Career
Neha Jhalani Hiranandani: A Guide to Her Life and Careerr98588472
 
Healthcare Feb. & Mar. Healthcare Newsletter
Healthcare Feb. & Mar. Healthcare NewsletterHealthcare Feb. & Mar. Healthcare Newsletter
Healthcare Feb. & Mar. Healthcare NewsletterJamesConcepcion7
 
Interoperability and ecosystems: Assembling the industrial metaverse
Interoperability and ecosystems:  Assembling the industrial metaverseInteroperability and ecosystems:  Assembling the industrial metaverse
Interoperability and ecosystems: Assembling the industrial metaverseSiemens
 
Entrepreneurial ecosystem- Wider context
Entrepreneurial ecosystem- Wider contextEntrepreneurial ecosystem- Wider context
Entrepreneurial ecosystem- Wider contextP&CO
 
digital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingdigital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingrajputmeenakshi733
 
Data Analytics Strategy Toolkit and Templates
Data Analytics Strategy Toolkit and TemplatesData Analytics Strategy Toolkit and Templates
Data Analytics Strategy Toolkit and TemplatesAurelien Domont, MBA
 
Who is the Darshan Hiranandani Wife ????
Who is the Darshan Hiranandani Wife ????Who is the Darshan Hiranandani Wife ????
Who is the Darshan Hiranandani Wife ????surajkumarsk121223
 
How the Insurance Industry is Scaling with AI
How the Insurance Industry is  Scaling with AIHow the Insurance Industry is  Scaling with AI
How the Insurance Industry is Scaling with AIPeter Ward
 
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...SOFTTECHHUB
 
Paul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky
 
Planetary and Vedic Yagyas Bring Positive Impacts in Life
Planetary and Vedic Yagyas Bring Positive Impacts in LifePlanetary and Vedic Yagyas Bring Positive Impacts in Life
Planetary and Vedic Yagyas Bring Positive Impacts in LifeBhavana Pujan Kendra
 

Recently uploaded (20)

WSMM Technology February.March Newsletter_vF.pdf
WSMM Technology February.March Newsletter_vF.pdfWSMM Technology February.March Newsletter_vF.pdf
WSMM Technology February.March Newsletter_vF.pdf
 
5 Digital Onboarding Trends To Watch In 2024.pdf
5 Digital Onboarding Trends To Watch In 2024.pdf5 Digital Onboarding Trends To Watch In 2024.pdf
5 Digital Onboarding Trends To Watch In 2024.pdf
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf
 
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdftrending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
 
Customizable Contents Restoration Training
Customizable Contents Restoration TrainingCustomizable Contents Restoration Training
Customizable Contents Restoration Training
 
Welding Electrode Making Machine By Deccan Dynamics
Welding Electrode Making Machine By Deccan DynamicsWelding Electrode Making Machine By Deccan Dynamics
Welding Electrode Making Machine By Deccan Dynamics
 
Pitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deckPitch Deck Teardown: Xpanceo's $40M Seed deck
Pitch Deck Teardown: Xpanceo's $40M Seed deck
 
Certificate of Good Standing Ferber Enterprises USA LLC
Certificate of Good Standing Ferber Enterprises USA LLCCertificate of Good Standing Ferber Enterprises USA LLC
Certificate of Good Standing Ferber Enterprises USA LLC
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers reference
 
Neha Jhalani Hiranandani: A Guide to Her Life and Career
Neha Jhalani Hiranandani: A Guide to Her Life and CareerNeha Jhalani Hiranandani: A Guide to Her Life and Career
Neha Jhalani Hiranandani: A Guide to Her Life and Career
 
Healthcare Feb. & Mar. Healthcare Newsletter
Healthcare Feb. & Mar. Healthcare NewsletterHealthcare Feb. & Mar. Healthcare Newsletter
Healthcare Feb. & Mar. Healthcare Newsletter
 
Interoperability and ecosystems: Assembling the industrial metaverse
Interoperability and ecosystems:  Assembling the industrial metaverseInteroperability and ecosystems:  Assembling the industrial metaverse
Interoperability and ecosystems: Assembling the industrial metaverse
 
Entrepreneurial ecosystem- Wider context
Entrepreneurial ecosystem- Wider contextEntrepreneurial ecosystem- Wider context
Entrepreneurial ecosystem- Wider context
 
digital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingdigital marketing , introduction of digital marketing
digital marketing , introduction of digital marketing
 
Data Analytics Strategy Toolkit and Templates
Data Analytics Strategy Toolkit and TemplatesData Analytics Strategy Toolkit and Templates
Data Analytics Strategy Toolkit and Templates
 
Who is the Darshan Hiranandani Wife ????
Who is the Darshan Hiranandani Wife ????Who is the Darshan Hiranandani Wife ????
Who is the Darshan Hiranandani Wife ????
 
How the Insurance Industry is Scaling with AI
How the Insurance Industry is  Scaling with AIHow the Insurance Industry is  Scaling with AI
How the Insurance Industry is Scaling with AI
 
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...
How To Simplify Your Scheduling with AI Calendarfly The Hassle-Free Online Bo...
 
Paul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate ProfessionalPaul Turovsky - Real Estate Professional
Paul Turovsky - Real Estate Professional
 
Planetary and Vedic Yagyas Bring Positive Impacts in Life
Planetary and Vedic Yagyas Bring Positive Impacts in LifePlanetary and Vedic Yagyas Bring Positive Impacts in Life
Planetary and Vedic Yagyas Bring Positive Impacts in Life
 

Lean analytics: Five lessons beyond the basics

  • 1. www.leananalyticsbook.com @leananalytics @byosko | @acroll Lean Analytics Use data to build a better business faster.
  • 2. A (really) quick intro to Lean principles
  • 3. The basic Lean message is learn and adapt, fast.
  • 4. Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
  • 5. In a startup, the purpose of analytics is to iterate to a product/market fit before the money runs out.
  • 6. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  • 7. The five Stages of Lean Analytics Empathy Stickiness Virality Revenue Scale Thestageyou’reat E-commerce SaaS Media Mobile app User-gen content 2-sided market The business you’re in One Metric That Matters.
  • 8. If it won’t change how you behave, it’s a bad metric. http://www.flickr.com/photos/circasassy/7858155676/
  • 9. Five lessons beyond the basics • Mining your existing data • A new product every day • Are you at your local maximum? • It’s different when you’re big • Build analytics into how you operate
  • 10. Mining your existing data Moving beyond guesswork.
  • 11. Identify a key business problem, pick the OMTM, draw a line in the sand, and get started.
  • 12. Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI The Lean Analytics Cycle
  • 13. Do hosts withDo hosts with professionalprofessional photography getphotography get more business?more business? Do hosts withDo hosts with professionalprofessional photography getphotography get more business?more business? Airbnb experiments...
  • 14. Professional photography helps Airbnb’s business Gut instinct Concierge MVP 20 photographers in the field Test results Two to three times more bookings! Back to the beginning Use additional data to keep experimenting
  • 16. Hang on a second...
  • 17. Really? Professional photography helps Airbnb’s business Gut instinct
  • 18. Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data: find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI Remember this?
  • 19. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data: find a commonality Without data: make a good guess
  • 20. The only thing worse than bad feedback is no feedback at all. - Dave McClure, Startupfest 2012
  • 21. Because it gives you nothing to analyze.
  • 22. Virality stage: Circle of Moms finds an engaged market • Stage: Stickiness • Model: UGC • Launched as Circle of Friends in 2007, it was a way for small groups to interact atop Facebook’s platform; but when engagement wasn’t good enough, the founders decided to dig deeper.
  • 23. The problem: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction
  • 24. A new product every day Why cohorts matter more than you think.
  • 25. Segments, cohorts, A/B, and multivariates Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁ Cohort: Comparison of similar groups along a timeline. A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis: Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ JanJan FebFeb MarMar AprApr MayMay
  • 26. Why use cohorts? Here’s an example. January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 How about this one? January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9
  • 27. Two views of the same company Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 28. Are we at a local maximum? You need optimization and inspiration.
  • 29. The problem with local maxima (http://www.slideshare.net/bokardo/metricsdriven-design-4317168)
  • 30.
  • 31. The vertebrate eye is “backwards and upside down”. Light travels through cornea, lens, aqueous fluid, blood vessels, ganglion cells, amacrine cells, horizontal cells, and bipolar cells. The cephalopod eye is constructed the “right way out”, with the nerves attached to the rear of the retina. No blind spot. Extension of the brain Invagination of the head
  • 32.
  • 33. It’s different when you’re big Tilting at the corporate windmill.
  • 34. “A startup is an organization designed to search for a sustainable, repeatable business model.” - Steve Blank Remember when I said,
  • 35. “A big company is an organization designed to perpetuate a business model.” - Me
  • 36. In other words, if your job is change you have your work cut out for you.
  • 37. Span of control and the railroads The story of Daniel C. McCallum
  • 39.
  • 40.
  • 41. Barnes and Noble tried pretty hard. • $1B invested in Nook • $475M operating loss in April • CEO gone
  • 43.
  • 44. Since large organizations make data that perpetuates things, you have to willfully ignore common wisdom at the outset. http://www.flickr.com/photos/jb912/8940173789
  • 45. Disruption may not fit neatly into an existing part of the organization, so you need to incubate it internally for a while.
  • 46. Unlike a VC or startup, when the initiative fails the organization still learns.
  • 47. Big organizations might not innovate at speed—but they innovate at scale.
  • 48. The Lean Analytics lifecycle for an Intrapreneur Empathy Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conflict, resistance, contracts Scale Hand the baton to others gracefully Hating what happens to your baby Stage Do this Fear this Beforehand Get buy-in Political fallout
  • 49. A large organization needs a portfolio of investments, judged by different criteria. Treasury bonds Venture capital
  • 50. A large organization needs a portfolio of investments, judged by different criteria. Safe Predictable Evolutionary Achieve the business plan Risky Uncertain Revolutionary Discover the business model Times of peace Times of war
  • 51. Acquisition or disruption? Innovate fast: nothing to lose Innovate slowly: conform to brand, legislation, approvals Found P/M fit Found P/M fit Integration complexity, founder churn, indigestion Biz case Scale Scale Biz model VC pitch
  • 52. Moving from box to box • How businesses think about products or companies • Lean is about moving up and to the right
  • 53. Frame the project as a learning exercise; product creation is almost accidental. Learn it’s a failure Learn it’s a success Industry knowledge Industry knowledge Possible product Business plan metrics here Market Business model metrics here Lean product Lines in the sand here
  • 54. Build analytics into how you operate the business Three threes and the Problem-Solution Canvas
  • 55.
  • 56.
  • 57. Three threes Three assumptions What big bets are you making? •“People will answer questions” •“Organizers are frustrated with how to run conferences” •“We’ll make money from parents” •“Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) •Product enhancements •Marketing strategies Three experiments to run What are you going to learn today? •Feature tests •Continuous deployment •A/B testing •Customer survey
  • 58. Three threes Monthly Weekly Daily Board, investors, founders Executive team Employees Strategy Tactics Execution Three assumptions Three actions to take Three experiments to run
  • 59. Three threes Get more people Increase answer % Test better questions Change the UI Test timings Questions from peers Many people will answer questions Three assumptions Three actions to take Three experiments to run
  • 60. In the end, be subversive.

Editor's Notes

  1. We ’ ve built on Eric ’ s model, because we believe that companies naturally go through five stages of growth. First, the need to get inside their customers ’ heads Then they need to make sure a small group of users keeps using the product Then they need to make sure that word can spread Then they need to feed some of their revenues back into customer acquisition, accelerated by the virality they ’ ve built. Then they need to scale.
  2. THis is a simple test for whether a metric is real or bullshit.
  3. Mike Greenfield and his co-founders started Circle of Friends in September 2007, shortly after Facebook launched their developer platform. The timing was perfect: Facebook became an open, viral place to acquire users as quickly as possible and build a startup. There had never been a platform with so many users (Facebook had about 50 million users at the time) that was so open to reaching them.Circle of Friends was a simple idea: a Facebook application that allowed you to organize your friends into circles for targeted content sharing. Mike notes now that it was basically, “ Google+ for Facebook ” (before Google+ existed.)
  4. By mid-2008, Circle of Friends had 10 million users. Greenfield focused on growth above everything else. “ It was a land grab, ” he said. And Circle of Friends was clearly viral. But there was a problem. Too few people were actually using the product.According to Mike, less than 20% of circles had any activity whatsoever after their initial creation. “ We had a few million monthly uniques from those 10 million users, but as a general social network we knew that wasn ’ t good enough and monetization would likely be poor. ” So Mike went digging.
  5. SegmentationA segment is simply a group that shares some common characteristic. It might be users that run Firefox; or restaurant patrons who make a reservation rather than walking in; or passengers who buy a first-class ticket; or parents who drive a minivan.On websites, we can often segment visitors according to a range of technical and demographic information. Then, by comparing one segment to another, we can learn things. If, for example, visitors using the Firefox browser have significantly fewer purchases, we may want to perform additional testing to find out why. If we see that a disproportionate number of buyers are coming from Australia, we might survey them to discover why, then try to replicate that success in other markets.Segmentation works for any industry and any form of marketing, not just for websites. As we ’ ll see, segmenting mailing lists is an essential part of improving e-mail performance; but direct mail marketers have been segmenting for decades with great success. Even if you ’ re selling to enterprise customers, dividing your messages by target audience, industry, or stage in the sales cycle—and then testing the results of those messages—is crucial.Cohort analysisA second kind of test is cohort analysis. As you build and test your product, you ’ ll be iterating regularly. Users that join you in the first week will have a different experience from those that join later on. For example, all of your users might go through an initial free trial, usage, payment, and abandonment cycle. As this happens, you ’ ll make changes to your business model. As a result, the users who experienced the trial in month one may have a significantly different onboarding experience from those who experienced it in month five. How did that affect their churn? To find this out, we use cohort analysis.Each group of users is a cohort—participants in an experiment across their lifecycle. You can compare cohorts against one another to see if, on the whole, key metrics are getting better over time.
  6. Using cohorts is really, really important.
  7. Using cohorts is really, really important.
  8. This is a fundamental problem with evolutionary optimization. To illustrate, let me tell you why you ’ ll always have a blind spot. There ’ s a place in your eye where you can ’ t see things. The other eye compensates, and your brain fills in the rest, but you really can ’ t see part of what you look at.
  9. The eye has evolved in nature in many, many different forms, often independently. Our eyes and those of cephalopods like the octopus look similar, but have very different histories. The human eye is “ backwards ” —the infrastructure is actually in front of the retina. That means the optic nerve runs in front of the part that catches the light. This is because the human eye evolved as a growth of the brain. On the other hand, the octopus eye has the optic nerve behind the retina, because it evolved as a dent in the head. Which means octopuses don ’ t have a blind spot. But you ’ re not going to evolve a better eye, because the intermediate mutations would be worse.
  10. Imagine I gave an algorithm three wheels and asked it to optimize their configuration. It would arrive at some form of tricycle. But it wouldn ’ t go, “ hey, can I get a fourth wheel? ” That would be what a human would do. That ’ s a leap of faith.
  11. If you work in a company of any significant size, you owe your org chart to an enterprising General Superintendent of the railroad era named Daniel C. McCallum. In the 1850s, railroads were a booming business. Unfortunately for investors, they didn ’ t scale well. Small railroads turned a profit; big ones didn ’ t.McCallum noticed this, and divided his railroad into smaller sections, each run by subordinates who reported back a standard set of information he defined. McCallum ’ s line—as well as other lines that copied this approach—thrived. McCallum ’ s model, inspired by his time as a soldier and the regimented hierarchies he had learned there, was then applied to other fields.McCallum was the first management scientist, introducing controls, structure, and regulations in order to reduce risk and increase predictability at scale. Companies like Google and Apple know this, creating their own advanced research groups such as the Google X Lab.
  12. As World War II exploded across Europe, the United States realized it needed a way to counteract German advances in aviation—specifically, jet aircraft. The US military asked Lockheed Martin (then the Lockheed Aircraft Corporation) to build a jet fighter. Desperate times called for desperate measures: in a month, they had a proposal; less than six months later, working in a closely guarded circus tent, the first plane was built.This group became known as the Skunk Works, a title that ’ s synonymous with an independent, autonomous group charged with innovation inside a bigger, slower-moving organization. These groups are often immune to the restrictions and budget oversight that guides the rest of the company, and have the specific goal of working “ out of the box ” to mitigate the inertia of large companies.
  13. Beforehand: get buy-inBefore you start doing customer development, you need executive buy-in. This may be implicit—if it ’ s your job to try and find new opportunities. But once you think you ’ ve found an opportunity, you need explicit approval from an executive. You want to know where you are on the BCG box, and where you ’ re trying to go. And you need to know what metrics your progress will be judged by. You need to know what resources you have, and what rules apply to you. This is like a pre-nup: better signed before the wedding.At this stage, you ’ re defining your analytical strategy, and the “ lines in the sand ” against which you ’ ll be judged. These may be goals for the whole company, such as margins; or they may be a growth rate that ’ s considered success. You ’ ll also need to define how you will adjust these metrics based on what you learn.Empathy: find problems, don ’ t test demandOnce you start doing customer development, remember that you ’ re testing problems and solutions—not existing demand. If you ’ re truly disruptive, customers won ’ t tell you what they want. But they will tell you why they want it. In 2008, Swiffer creator Gianfranco Zaccai explained, “ Successful business innovation isn ’ t about giving consumers what they need now, but about giving them something they ’ ll desire in the future. ” Customers weren ’ t telling Netflix they wanted to stream videos, but their patterns of usage, computer adoption, broadband deployment, and browsing told the company a need existed.This is a place for qualitative interviews. You should talk to existing users and customers, of course. But if you ’ re trying to grow market share, you ’ ll also want to talk to your competitors ’ customers, to distributors, and to everyone involved in purchasing the product. If you ’ re trying to improve growth rate, you ’ ll talk to adjacent customers. That ’ s what Bombardier did when it expanded from its cash cow snowmobile business to its rapidly growing jet-ski business.Skip the business case, do the analyticsAt some point, when it comes time to go beyond interviewing people, you ’ ll need to build a business case. Traditional product managers build profit-and-loss analyses to try and justify their plans: they create a convincing business case, and once someone believes it, they get funding to proceed. But a Lean mindset reverses this: you sell the business model, without a lot of prediction, and then rely heavily on analytics to decide whether to kill the product or double-down on it.This analyze-after rather than predict-before model is possible because many of the costs of innovation can be pushed later in the product development cycle. Just-in-time manufacturing, on-demand printing, services that replace upfront investment with pay-by-the-drink capacity, CAD/CAM design, and mercenary contractors all mean that you don ’ t have to invest heavily up front (and therefore doesn ’ t have to argue a business case at the outset.) Rather, you can ask for a modest budget, build analytics into the product, and launch sooner for less money. You can then use the data and customer feedback you get, which is vanishingly cheap to collect given today ’ s technology, to plead your case based on actual evidence.Stickiness: know your real minimumIf you ’ ve identified a problem worth solving and a solution that customers will want, it ’ s time to make an MVP. But you need to know what your real minimum is. As a big organization, you may have restrictions on data sharing, reliability, or compliance to which smaller organizations (who have less to lose) aren ’ t subject. You also need to know what your unfair advantages are.Consider, for example, the many meal pre-ordering tools on the market today. These mobile applications let you place an order from a food court restaurant, pay, and pick up at an agreed-upon time without waiting. The restaurants like it because they save precious time in the lunchtime rush, and the diners like it because it ’ s simple and they can browse the menu at their leisure. It ’ s like Uber for lunch.Now consider what would happen if McDonalds were to decide to compete by introducing an application. They might have franchise constraints, or regulations for restaurants located in airports, or state laws about disclosing caloric content. All of these would have to be part of the MVP.Offsetting this, however, is the huge amount of market control the company has. It could promote the app, and give away three hamburgers for free to everyone who installed it. The company would make back the money quickly in saved time at the cash register, and have access to a new marketing channel and untapped analytical insight into its customers.Intrapreneurs need to factor these kinds of constraints and advantages into their MVP far more than independent startups do.What ’ s more, as people start to use your MVP, you have to manage the beta process carefully. You may be interfering with existing deals in the sales pipeline, or creating more work for customer support. If so, you need to have approval for the roll-out and the buy-in of stakeholders. If you ’ re launching an entirely new product line, you may even have to camouflage it so you don ’ t cannibalize existing markets until you know it ’ s successful. This, of course, undermines your ability to use unfair advantages like an existing customer base.Viral from the startIf you ’ re trying to move upwards in the BCG box, your product should include viral and word-of-mouth elements. In a world where everyone has access to a mobile device, every product needs to have an interactive strategy. There ’ s simply no excuse not to find a viral angle to act as a force multiplier for growth. In fact, adding a viral component is one of the keys to moving dogs and cash cows up into question marks and stars.Revenue within the ecosystemYou ’ ll have less flexibility to set pricing and reinvest revenues in product marketing, because as you grow you ’ ll have to coexist with other marketing efforts by your host company. When Microsoft wanted to test its SaaS-based office suite, it could do so in a relatively controlled way. But as soon as it wanted to monetize the product, it had to contend with cannibalization and push-back from a channel that depended on license revenue.Your pricing may have to take into account channels, distributors, and other factors that restrict your freedom to experiment, because changes you make will have an impact on other products in the marketplace. Had Blockbuster entered the streaming video market, it would have had to deal with labor and real estate issues at existing stores.Scale and the handoffIn the final stages of intrapreneur innovation, the new product has proven its viability. It ’ s either stolen by settlers—who can help it cross the chasm and broaden its appeal—or the team that created it must transition to a more traditional, structured model of business and take its place among the other products and divisions of the host organization.Most of the time, the DNA of a disruptive organization isn ’ t well suited to “ boring ” management and growth, so you ’ ll need to hand the product off to the rest of the organization and find the next thing to disrupt. That means you really have two customers: the external one buying the product, and the internal one that has to make, sell, and support it.Ultimately, the intrapreneur must manage the relationship with the host organization as well as the relationship with the target market. Initially, this can be intentionally distant, but as the disruptive product becomes part of the host, the hand-off must be graceful.
  14. Why might you not want to disrupt things? To understand this, you need to look at how large organizations plan their product and market strategy.The Boston Consulting Group (BCG) box, shown in Figure 54, is a simple way to think about a company ’ s product portfolio. This venerated model classifies products or subsidiaries according to two dimensions: how quickly the market is growing, and how big a market share the company has in that market.Products with high market share, but slow growth, are “ cash cows. ” They generate revenue, but they aren ’ t worthy of heavy investment. By contrast, products with high growth but small market share are “ question marks, ” candidates for investment and development. Those with both growth and market share are the rising “ stars ” . Those with neither—called “ dogs ” —are to be sold off or shut down.The BCG box offers a thumbnail of a company ’ s product portfolio. It ’ s also a good way to think about innovation. If you ’ re trying to change a company, you ’ re either trying to create a new product (hopefully in a growing market) or you ’ re trying to innovate to revitalize an existing product with the addition of new features, markets, or services.
  15. The first thing you ’ ll notice is the title: the Goal is to Learn. This is important, because it reminded the entrepreneurs about what they were setting out to do. It wasn ’ t about building “ stuff. ” It wasn ’ t about adding features. It wasn ’ t about getting PR, or anything else. Learning was the measure of success.Next, they would fill in a brief update on their current status, focusing on the key metrics (qualitative and/or quantitative) that they were tracking. Notice how small this box is compared to the others.The Lessons Learned box is a quick bulleted summary of key learning. The title says “ and accomplishments ” because we wanted to give entrepreneurs a place to brag … at least a little bit. Not surprisingly, they ’ d include some vanity metrics in here and we wouldn ’ t focus a lot of time on them. The “ on track: yes/no ” benchmark is designed as a test of intellectual honesty. Can entrepreneurs really come clean on what ’ s going on, good and bad? If so, we could be much more valuable.Finally, we asked entrepreneurs to list the top problems they were facing at that moment. At most we ’ d include three problems prioritized in order of importance. This section of the Problem-Solution Canvas often elicited the most debate; but it was always healthy and critical for resetting everyone ’ s goals and expectations.
  16. In this section, the founders re-list the problems, and include hypothesized solutions. These solutions are hypothesized because we don ’ t know if they ’ ll work. These are experiments that the founders will run in the next week. We always asked them to define the metrics they ’ d use to measure success (or failure) and draw a line in the sand. If engagement was the most important problem, they had to include possible solutions they ’ d experiment with to increase engagement, define the metric (example: % daily active users) and set a target. What ’ s the problem, how do you propose to fix it, and how will you know if you succeeded? That ’ s the core of the Problem-Solution Canvas.For us (as mentors and advisors) it was an extremely valuable exercise. The Problem-Solution Canvas is also useful for internal decision-making. It ’ s a level below Lean Canvas, focusing on very specific details in a very specific time period (1-2 weeks.)
  17. Here ’ s an example of finding a cause. SOME years ago, executives at a Houston airport faced a troubling customer-relations issue. Passengers were lodging an inordinate number of complaints about the long waits at baggage claim. In response, the executives increased the number of baggage handlers working that shift. The plan worked: the average wait fell to eight minutes, well within industry benchmarks. But the complaints persisted.Puzzled, the airport executives undertook a more careful, on-site analysis. They found that it took passengers a minute to walk from their arrival gates to baggage claim and seven more minutes to get their bags. Roughly 88 percent of their time, in other words, was spent standing around waiting for their bags.So the airport decided on a new approach: instead of reducing wait times, it moved the arrival gates away from the main terminal and routed bags to the outermost carousel. Passengers now had to walk six times longer to get their bags. Complaints dropped to near zero. Turns out that long delivery wasn ’ t causing dissatisfaction. Time spent waiting was.