Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Lean Analytics
New Media Manitoba
June, 2015
@acroll
I can’t tell you how to succeed
(If I could, I’d be starting your company instead of standing here.)
I can tell you how to avoid failure
(At least, some forms of it.)
Percent of businesses that fail
Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10
71%69%66%63%60%55%5...
Still operating after 4 years
Finance Insurance and Real Estate
Education and Health
Agriculture
Services
Wholesale
Mining...
Why firms die
http://www.statisticbrain.com/startup-failure-by-industry/
Neglect,

fraud,

disaster

1%
Lack of industry

e...
Part one:

Introduction to Lean Analytics
Don’t sell what you can make. Make what you can sell.
Kevin Costner is a lousy entrepreneur.
The core of Lean
is iteration.
Waterfall approach
You know the problem and the solution.
Known set of
requirements
Known ways to satisfy
them
Spec Build Test Launch
Agile methodologies
Know the problem, find the solution
Known set of
requirements
Unclear how to satisfy
them
Build Test LaunchViable?Problem

statement
Adjust
Sprints
Unknown se...
Lean approach
First, know that you don’t know.
Possible problem
space
Product/
market
hypothesis
Trial startup
Product/
market
hypothesis
Trial startup
Product/market
hy...
Unfortunately,
we’re all liars.
Everyone’s idea is
the best right?
People love
this part!
(but that’s not always
a good thing)
This is where
things fall a...
Most startups don’t know what they’ll
be when they grow up.
Hotmail

was a
database
company
Flickr

was going to
be an MMO...
Gottfried von Leibniz
was a geek just like you.
First calculator

(stepped reckoner)
One of the first
to recognize the
importance of
binary.
“I thought again
about my earl...
The best of all possible
worlds is the one in
which the fewest
starting conditions
produce the greatest
variety of outcome...
Analytics can help.
Analytics is the measurement of
movement towards your business
goals.
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
Why now?
First: High rate of change of digital
technologies & channels.
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
0
15
30
2007 2012
266
Some fundamentals.
Analytics, performance, aggregation, and the right metrics
We’re bad at data.
I have two kids.
At least one of them is a girl.
What are the chances
the other is a boy?
BB BG
GB GG
2 of 3 (66%) are boys.
GB GG BG
Fundamental:

What makes a good metric?
A good metric is:
Understandable
If you’re busy
explaining the
data, you won’t
be busy acting
on it.
Comparative
Compariso...
The
simplest
rule
bad

metric.
If a metric won’t change how
you behave, it’s a
h"p://www.flickr.com/photos/circasassy/78581...
Metrics help you know yourself.
Acquisition
Hybrid
Loyalty
70%

of retailers
20%

of retailers
10%

of retailers
You are
j...
Qualitative
Unstructured, anecdotal,
revealing, hard to
aggregate, often too
positive & reassuring.
Warm and fuzzy.
Quanti...
Exploratory
Speculative. Tries to find
unexpected or
interesting insights.
Source of unfair
advantages.
Cool.
Reporting
Pre...
Rumsfeld on Analytics
(Or rather, Avinash Kaushik channeling Rumsfeld)
Things we
know
don’t

know
we know Are facts which ...
MayAprMarFeb
Slicing and dicing data
Jan
0
5,000
Activeusers
Cohort:
Comparison of
similar groups
along a timeline.
(this ...
Which of these two companies
is doing better?
  January February March April May
Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50
Is this company
growing or stagnating?
Cohor...
Cohort 1 2 3 4 5
January $5 $3 $2 $1 $0.5
February $6 $4 $2 $1  
March $7 $6 $5    
April $8 $7      
May $9        
Avera...
Lagging
Historical. Shows you
how you’re doing;
reports the news.
Example: sales.
Explaining the
past.
Leading
Forward-loo...
Fundamental:

Correlation
1
10
100
1000
10000
Ice cream consumption Drownings
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Correlated
Two variables that are
related (but may be
dependent on
something else.)
Ice cream &
drowning.
Causal
An indepe...
A leading, causal metric
is a superpower.
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Is social action a leading
indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-fu...
Is mobile use?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
Soundcloud
Cory Levinson, Soundcloud
A Facebook user reaching 7 friends within 10 days of signing up
(Chamath Palihapitiya)
If someone comes back to Zynga a da...
AirBnB and Craigslist
http://www.flickr.com/photos/bootbearwdc/1243690099/
Think subversively.
Why is Nigerian spam so badly
written?
Aunshul Rege of Rutgers University, USA in 2009
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 me...
Turns out the word “Nigeria” is the best
way to identify promising prospects.
Nigerian spammers
really understand their target market.
They see past vanity metrics.
Fundamental:
KISS
“It can scarcely be denied
that the supreme goal of all theory is
to make the irreducible basic elements
as simple and as ...
“As simple as possible,

but no simpler.”
*(FYI, this is irony.)
The Lean Analytics framework.
Eric’s three engines of growth
Virality
Make people
invite friends.
How many they
tell, how fast they
tell them.
Price
Spe...
Dave’s Pirate Metrics
AARRR
Acquisition
How do your users become aware of you?
SEO, SEM, widgets, email, PR, campaigns, bl...
Stage
EMPATHY
I’ve found a real, poorly-met need that a
reachable market faces.
STICKINESS
I’ve figured out how to solve th...
Do you have a business model?
http://www.flickr.com/photos/tinfoilraccoon/197640807/
A lemonade stand’s business model is simple.
Yours should be too.
A hobby.
PhotobyRakkaonFlickr.https://www.flickr.com/photos/rakka/6889638004
Market Sizing
Top-down and bottoms-up
Top-down
Start with the whole planet, then narrow it down.
Population of Canada
Percent mobile phones by country
IOS penetration
Put this all together
Metric/Assumption Value
Population of Canada 34.88M
Mobile phone percentage 80%
Mobile phones in Can...
How much is the TAM worth?
Metric/Assumption Value
TAM 1.39M
Percent we will claim 5%
Number of users 69,500
User lifetime...
Bottoms-up
Start with one buyer, then add it up
Bottoms-up
Metric/Assumption Value
Impressions (acquisition) 9,266,667
Percent that install 10%
Installs (activation) 926,...
Reality check
Bottoms-up (TAM) Value
Impressions (acquisition) 9.3 M impressions
Top-down (TAM) Value
IOS accounts we can ...
Six business model archetypes.
E-commerce SaaS Media
Mobile

app
User-gen

content
2-sided

market
The business you’re in
Can you move enough
customers through this
cycle to make more money
than you spend?
Make them
refer others
Retain them
Get...
(Which means eye
charts like these.)
Customer Acquisition Cost
paid direct search wom
inherent
virality
VISITOR
Freemium/t...
Model + Stage = One Metric That Matters.
One Metric

That Matters.
The business you’re in
E-Com SaaS Mobile 2-Sided Media ...
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics
SaaS-based SEO toolkit in the scale stage. Focused on net adds.
Was a marketing campaign successf...
Metrics are like squeeze toys.
http://www.flickr.com/photos/connortarter/4791605202/
Empathy
Stickiness
Virality
Revenue
Scale
E-
commerce
SaaS Media
Mobile

app
User-gen

content
2-sided

market
Interviews;...
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its
customers every year.
Is this good or bad?
Not knowing what normal is
makes you do stupid things.
Baseline:
5-7% growth a week
“A good growth rate during YC
is 5-7% a week,” he says. “If
you can hit 10% a week you're
doi...
It’s oxygen
You need customers to keep learning
It’s a substitute for solvency
PhotobyPaulMilleronFlickr.https://www.flickr...
Baseline:
10% visitor engagement/day
Fred Wilson’s social ratios
30% of users/month use web or mobile app
10% of users/day...
Baseline:
2-5% monthly churn
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s
on the job.
• Get below a 5%...
Baseline:
Calculating customer lifetime
25%

monthly churn
100/25=4

The average
customer lasts
4 months
5%

monthly churn...
Baseline:
CAC under 1/3 of CLV
• CLV is wrong. CAC Is probably wrong, too.
• Time kills all plans: It’ll take a long time ...
Who is worth more?
Today
A
Lifetime:
$200
Roberto Medri, Etsy
B
Lifetime:
$200
Visits
The Lean Analytics cycle
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in productio...
The minimum needed to

test the core idea
Survey owner adds recipient to group
Survey owner asks question
Recipient reads ...
Do AirBnB hosts
get more business
if their property is
professionally
photographed?
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
Candidate solution (MVP)
20 field photographers ...
5,000 shoots per month
by February 2012
Hang on a second.
Gut instinct (hypothesis)
Professional photography helps AirBnB’s business
SRSLY?
Draw a new line
Pivot or

give up
Try again
Success!
Did we move the
needle?
Measure
the results
Make changes
in productio...
“Gee, those
houses that do
well look really
nice.”
Maybe it’s the
camera.
“Computer: What
do all the
highly rented
houses ...
Circle of Moms: Not enough engagement
• Too few people were
actually using the
product
• Less than 20% of any
circles had ...
Landing page design A/B testing
Cohort analysis General analytics
URL shortening
Funnel analytics
Influencer Marketing
Publ...
Part two:

Why big companies

need to change
(http://csinvesting.org/2012/01/06/fortune-500-extinction/)
F500 Life
Expectancy
Growth by entering
a new business 95

% f...
mikemace.com
The slow
death of
a market
leader.
Revenue over time

This is what most
managers track.
Note that sales keep
...
In other words, if your job is change you
have your work cut out for you.
Lesson one:
Companies die because they fail to
move to new business models.
Clay Christensen, The Innovator’s Dilemma
CostperMB
$1000
$100
$10
$1
Time
14”
M
ainfram
e
8”
M
inicom
puter
5.25”
Desktop...
Technologies
outstrip what
the market
needs, driven
by feedback
from the
“best”
current
customer.
Clay Christensen, The In...
The new
market has
different criteria
for success,
which are
uninteresting to
incumbents.
Clay Christensen, The Innovator’...
Sometimes
this has
unintended
consequences
Clay Christensen, The Innovator’s Dilemma
$1000
$100
$10
$1
Time
Smaller disc s...
Why now?
First: High rate of change of digital
technologies & channels.
Arbitron and radio data
Times a song in “heavy
rotation” is played daily
2007 2012
266
This alone explains the collapse of
modern print media.
Circulation, annually Clicks, instantaneously
Letters to the edito...
Why now?
Second: It’s no longer about whether
you can build it—it’s about whether
anyone will care.
The Attention Economy
“What information consumes
is rather obvious: it consumes
the attention of its recipients.
Hence a w...
Lit motors tests the risky part
Lesson two:
The difference between a rogue agent
and a special operative is permission.
If big firms
can’t innovate,
it’s this guy’s
fault.
Technology has radically changed the
incremental cost of businesses.
http://www.flickr.com/photos/ebolasmallpox/3733059220/
Software is eating the world.
An economic order quantity
of one.
Crafted
Mass-
produced
Automated Digital
Quantity Few Many Some One
Cost High Low Mediu...
Sustainable competitive advantage allows for
inertia and power to build up along the lines of
an existing business model, ...
Scale is now a liability. Compete on
cycle time.
CW&T make a pen
http://www.flickr.com/photos/art_es_anna/288880795/
Optimizing the probable means
discounting the possible.
This isn’t about a lack of resources.
http://www.flickr.com/photos/maladjusted/5207565912
Blockbuster had a lot going for it.
Plenty of inventory, of course.
But that matters less than...
...market intelligence,
customers, existing
payment approval,
and customer
history.
The problem was framing:
Blockbuster thought it was in the video
store management business. Netflix
realized it was in the ...
YOU ARE
HERE
YOU ARE
HERE
LOCAL

MAXIMUM
OPTIMIZATION

OF CURRENT
METRICS
YOU ARE
HERE
GLOBAL

MAXIMUMINNOVATION

WITH NEW

RULES
YOU ARE
HERE
SHORT-TERM INVESTORS

HATE GOING DOWNHILL
• $1B invested in Nook
• $475M operating loss
in April 2013
• CEO gone
First mover advantage happens
long before the marke...
Constraints slow things down
vs.
Capital cycles don’t fit the short, iterative
nature of startup uncertainty
12 month budgeting cycle; annual plan. Future b...
What’s the
biggest
problem

in a hair
salon?
The empty chair.
Knowing this, how
might you change
the client/customer
relationship?
Can and must
What triggers cultural change?
“Using Cue, you can tell if someone
has the ‘flu in 10 minutes.”
http://tiltthewindmill.com/when-can-becomes-must/
“Using Cue, you must tell if someone
has the ‘flu in 10 minutes…
…or Johnny can’t come on the school trip.”
Part three:
Three maxima of innovation
When you’re a startup
your goal is to find a sustainable,
repeatable business model.
When you’re a big company
your goal is...
Intrapreneur:
Someone working to produce
disruptive change in an organization
that has already found a sustainable,
repeat...
Business model vs.
company stage
Company size/age
Early stage Big/incumbent
B2B
Target

market
B2C
LessWoM

Moreformaldeci...
In a startup, the purpose of analytics is
to iterate to product/market fit
before the money runs out.
In a big company,
analytics replaces opinion with fact.
Companies that use data-driven
analytics instead of intuition have
5%-6% higher productivity and
profits than competitors.
...
The fundamental shift from Big Data
Ask
question
Define

schema
Collect

data
Answer

question
Refine
problem
Collect
data
A...
Three kinds of innovation
Sustain/core

(optimizing for more of the same)
Innovate/adjacent

(introduce nearby product,

m...
Many models for enterprise innovation
Core Adjacent Transformative
Do the same
thing better.
Nearby product,
market, or me...
Another way to look at it
Core Adjacent Transformative
Know the problem
(customers tell you it)
Know the solution
(custome...
Experiment with product, market,
and method.
Product
(new “what”)
Market
(new “who”)
Method
(new “how”)
3 kinds of
innovation
Engine as a service
http://www.nasa.gov/images/content/365835main_airplane_noise_qtd2_3024x2016.jpg
Engine as a service
“Efficiency is tied to
analytics. We’ll still look
for new materials, or for
the physics of devices,
but the analytics ......
Current

state
Business
optimization
(five mores)

Product,

market,

method
innovation

Business
model
innovation

You can...
Improvement Adjacency Remodeling
Do the same,

only better.
Explore what’s

nearby quickly
Try out new

business models
Le...
Sustaining Adjacent Disruptive
Next year’s car Electric car,

same dealer
On-demand, app-based

car service
Sustaining
innovation
is about
more of
the same.
(says Sergio Zyman)
More things
To more people
For more money
More often
...
Blizzard extends the
lifespan of WOW
Early

adopters
Rapid

growth
Market

saturation
The infamous S-curve
(Product lifecy...
Blizzard extends the
lifespan of WOW
Blizzard extends the
lifespan of WOW
Fixing this: sustaining growth with novelty
Product & market innovation
(“New & impro...
Blizzard extends the
lifespan of WOW
WOW
Burning

Crusade
Wrath of

the Lich King
Mists of

PandariaCataclysm
Warlords of
...
Most of your
innovation will
be adjacent or
sustaining.
Question marks!
(low market share,
high growth rate)
May be the ne...
Software, experimentation, and
iterative cycles of learning help you
get to the local maximum better and
faster. That’s a ...
Adjacent innovation is about changing
one part of the model in a way that
alters the value network.
Amazon Web Services and the
server value network
Server computing
• Density
• Heat
• GHz
• MIPS
Cloud computing
• Instance...
Adjacent product
to the same
market in the
same way
Selling the same product to an
adjacent market in the same
way.
Of P&G’s 38 brands, only 19 were sold in Asia as of 2011
M...
Selling the same product to the same
market in a new way.
The biggest innovation in
logistics of the 20th century.
http://...
Changing the method of
C2C classifieds
A blend of who, what, how
Classified C2C sales (same
“what”)
Strictly for Japanese wo...
Selling the same product to the
same market in a new way.
(At this point, observant Intrapreneurs
should be asking, should P&G be in
the house cleaning business?
And that would be ...
Transformative innovation is about
taking a leap, changing more than one
dimension simultaneously in search of
a new busin...
If sustaining, incremental innovation
produces linear growth, then
disruptive, transformative innovation
produces exponent...
Transformative isolation:
Skunkworks
Transformative
incubation:
Metlife Infinity
Significant market
850K full-time law enforcement officers in the
US; 700K state/local; 525K patrol officers
130M incident re...
Part four:
What works for large companies
(A bagful of tricks from agitators in companies of all sizes.)
The job of an intrapreneur is to
identify an adjacent market, product,
or method that conforms to
organizational filters.
I...
Also: a pariah.
Successful innovators share certain attributes.
Bad listener: Wilfully ignore feedback from your best cust...
Know what
kind of
innovation
you’re
after.
New
Current
Current New
Market
Product
Penetrate:

Increase revenues,
market sh...
Innovation portfolios at big companies
Core Adjacent Transformative
70% 20% 10%
Investment
70%20%10%
Return
Use outliers and missed searches to
hunt for good ideas & adjacencies
(Multi-billion-dollar hygiene product company)
1/8 m...
Frame it like a study
Product creation is almost
accidental.
Unlike a VC or startup, when
the initiative fails the
organiz...
When in doubt, collect data
From tackling the FTA rate to
visualizing the criminal justice
supply chain.
Use data to create a taste for
data
Sitting on Billions of rows of
transactional data
David Boyle ran 1M online surveys
On...
4” e-ink display with
name and specialty.
Badge scans barcode and
gets specs; checks
inventory; enters data on a
touch scr...
Don’t just collect data,
chase it.
Understand hidden
constraints
That pencil story is a myth.
Graphite is conductive and
explosive. The Minimum
Viable Produc...
Know what has to
be built in-house
SAP integration
Employment law
http://www.flickr.com/photos/bootbearwdc/1243690099/
Think subversively.
Everything’s an excuse to experiment
Find other ways
to collect data;
everything is an
experiment.
Run it as a consulting business first.
(Just don’t get addicted to it. Your goal is to
learn and overcome integration chall...
Convince your boss she asked for this
Draw a new line
Pivot or

give up
Try again
Success!
Did we move
the needle?
Measure...
Focus on the desired behavior, not just
the information.
http://www.psychologytoday.com/blog/yes/
200808/changing-minds-an...
Take baby steps.
Netflix
Tesla
http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character
limit isn’t arbitrary. It’s
constrained by the size
of SMS (160
characters) and
username (20
chara...
Figure out how to translate it back to a
simple model that fits the company’s
existing value model.
If your company dies, t...
Software
Platform
Merchandising
User-generated content
Marketplace
Media/content
Service
Oracle’s accounting suite
Amazon’...
Acquisition

channel
Selling

tactic
Revenue

model
Product

type
Delivery

model
Business
aspect
Flipbook

page(s)
Inhere...
Use the right proxies for B2B products
Stage Startup metrics Intrapreneur metrics
Empathy
Customers interviewed (needs &
s...
Tomorrow’s company: Running parallel businesses
Innovation Sustaining/core Adjacent
Transformative/
disruptive
Core action...
How to build a message map
I need a carA.
I should buy

a car
B.
It should be

a hybridC.
I should buy

a Honda CivicD.
Everyone in the world
People who want to drive
Prospective car buyers
People looking for a hybrid
Honda Civic Hybrid owners
I need a carA.
I sho...
“Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads...
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I wan...
People who want to drive
“I need a vehicle to get
around, be productive, and
enjoy my life.”
Prospective car buyers
“I wan...
“Isn’t it time you got out of the
city?” campaign showing how cars
make nature accessible & ridiculing
urban hipsters.
Ads...
Hits
A metric from the early, foolish days of the Web.
Count people instead.
Page views
Marginally better than hits. Unles...
Example: a restaurant
• Empathy: Before opening, the owner first learns about the diners in its area, their
desires, what f...
Example: a software company
• Empathy: The founder finds an unmet need, often because she has a background in a particular
...
Empathy
Get inside their head.
How to avoid leading the witness
Don’t tip your hand
Avoid biased wording, preconceptions,
or a giveaway appearance. Word ...
Mobile app model:

Localmind hacks Twitter
• Stage: Empathy
• Model: UGC/mobile
• Real-time question and answer platform t...
Localmind hacks Twitter
• Before writing a line of code, Localmind was concerned that people would never
answer questions....
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
Ask what existing
brands come to
...
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey Demographic
s...
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (yo...
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (you’ll always have mistakes)
• Send it o...
Creating an answers-at-scale campaign
• Know why you’re doing a survey in the first place
• Design the survey
• Test it (yo...
When it’s time to move on
• Have you conducted enough quality customer interviews to feel confident that
I’ve found a probl...
Stickiness
The dogs like the dogfood.
Hits1995
Visits1997
Visitors1999
Conversions2002
Engagement2010
Who did you add? Where from? Why?
What did they do? How di...
Days since last visit
1200
1000
800
600
400
200
0
1 2 3 4 5 6 7 8 9
Days since last engagement
January February
Disengaged...
When it’s time to move on
•	 Are people using the product as expected?
•	 Define an active user. What percentage of your us...
Virality
I told two friends.
------------------------------------------------------

Get your free private email at http://www.hotmail.com

-----------...
v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0 
http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/
Viral co...
Or simpler
Users
Viral

coefficient
Churn &

abandonment
x - >
1
How to calculate it
• First calculate the invitation rate, which is the number of invites sent divided by the
number of us...
Virality stage:

Timehop’s content sharing
• Stage: Virality
• Model: Mobile app
• Social network around the past
• Focuse...
The one metric that matters: content
sharing
• Focused on percent of daily active users that share their content
• Aiming ...
3 kinds of virality
•	 Inherent virality is built into the product, and happens as a function of use.
•	 Artificial viralit...
Laid, paid, made, afraid
Who is your
audience?
Drive-by visitors vs. serious evaluators; loyal users vs.
one-time buyers; ...
When it’s time to move on
• Are you using one of the three types of virality (inherent, artificial, word of mouth)
for your...
Revenue
Pour some of the money back into acquisition.
What’s a fitbit customer worth?
• The user can record their steps with a device in their pocket
• They can use it and sync ...
When it’s time to move on
You’re making money
You’re sustainable
You’re tracking growth metrics
Revenue stage:

Backupify’s customer
lifecycle
• Stage: Revenue
• Model: SaaS
• Leading backup provider for cloud based da...
You need a business model
Start with a systems diagram
What’s the core job to be done?
Start with the customer journey.
Consider the morning routine.
Wake
up
What happens here?
Start
workday
What job are you doing?
Personal
dimension
Social
dimension
Functional
aspects
Emotional
aspects
Related jobs to
be done
F...
Changes create new JTBD answers
Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953
+ = ?
Understand culture
What will it take for a change to really stick?
Whither

Estonia?
How about now?
Some stats
• 215M digital signatures
• 18 minutes to start a company, entirely online
• 1/3 of voting happens via the Inte...
Get rid of the pens:

Digital signatures since 2000.
Only collect things once:

Tax forms 95% completed already.
Conclusions
“The most important figures that one
needs for management are unknown
or unknowable, but successful
management must neverth...
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES
HAD TAKEN
BATHS BEFORE.
Once, a leader convinced others
in the absence of data.
Now, a leader knows
what questions to ask.
Alistair Croll
acroll@gmail.com
@acroll
Ben Yoskovitz
byosko@gmail.com
@byosko
Day two business models
Hardcore interviews
Maybe they don’t love you

like they said they do.
N
Your offering doesn’t make them want to

brag or their contact isn’t ...
B2B and the 80/20 rule
How to tell if you have a sustainable, repeatable B2B idea.
Target customer
Reachable, referenceable.
Find 10 prospects*
Target customer
Reachable, referenceable.
Target customer
Rea...
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
St...
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
Standardized
(sameprocesses,docs,tools)
St...
Standardize, automate, patentCustom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Custom
Your margins,
differenti...
The whole point of digital is personal
Segment 1
User segment

(who)
Segment 2
Segment 3
Goal

(what)
Goal 1
Goal 2
Goal 3...
http://www.thirdwunder.com/funnels-are-a-horrible-metaphor/Photo Credit: Patrick McGarvey
More like this.
The leaky bucket of subscriptions.
Photo by Jeffrey on Flickr - https://www.flickr.com/photos/jb912/7479464134
Bounces Non-

creators
Non-

payers
Churn
Signup

rate
Engagement

level
Conversion

rate
Visitors
Subscribers
Engaged use...
Bounces Non-

creators
Non-

payers
Churn
Improve stickiness; call-to-action optimization;

A/B page testing; picking bett...
Let them ask for an out
Detecting SaaS churn early without hurting cashflow
The tradeoff
Charge a monthly fee Charge annual fee up front
Find out if they hate it sooner, when
they cancel on first bil...
They’re happy, you keep your
money, goodwill for offering.
They’re unsatisfied, you made it
right, they tell you why, you l...
The mobile app!
customer lifecycle!
Ratings
Reviews
Search
Leaderboards
Purchases
Downloads
Installs
Play
Disengagement
Re...
Targeting
matters.
It’s all about cabbages
Start with the customer journey.
Consider the morning routine.
Wake
up
What happens here?
Start
workday
What job are you doing?
Personal
dimension
Social
dimension
Functional
aspects
Emotional
aspects
Related jobs to
be done
F...
Changes create new JTBD answers
Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953
+ = ?
I lied. Everyone is a tech company.
http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326
Cost of experiments...
Let’s pick on restaurants
for a while.
A line in the sand
Labor costs
Gross revenue
30%
24%
20%
Too costly?
Just right
Understaffed?
=
A leading indicator
http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970
50 r...
http://www.flickr.com/photos/southbeachcars/6892880699
Restaurant MVP
Is tip amount a leading indicator of long-
term revenue?
Why does every table get the same
menu?
Is purple ink better?
http://tippingresearch.com/uploads/managing_tips.pdf
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Slides from New Media Manitoba Lean Analytics workshop, June 2015
Upcoming SlideShare
Loading in …5
×

Slides from New Media Manitoba Lean Analytics workshop, June 2015

7,800 views

Published on

Slides from an all-day workshop with Winnipeg-based startups.

Published in: Data & Analytics
  • Profollica�'s all-natural formula helped 90% of men reduce hair loss in a clinical trial. ■■■ http://t.cn/AiHip2fH
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Have you ever heard of taking paid surveys on the internet before? We have one right now that pays $50, and takes less than 10 minutes! If you want to take it, here is your personal link ▲▲▲ https://tinyurl.com/make2793amonth
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Earn $500 for taking a 1 hour paid survey! read more... ♣♣♣ http://ishbv.com/surveys6/pdf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • How can I get a flat stomach without exercise? ●●● https://tinyurl.com/bkfitness4u
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Have you ever heard of taking paid surveys on the internet before? We have one right now that pays $50, and takes less than 10 minutes! If you want to take it, here is your personal link ■■■ https://tinyurl.com/make2793amonth
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Slides from New Media Manitoba Lean Analytics workshop, June 2015

  1. 1. Lean Analytics New Media Manitoba June, 2015 @acroll
  2. 2. I can’t tell you how to succeed (If I could, I’d be starting your company instead of standing here.)
  3. 3. I can tell you how to avoid failure (At least, some forms of it.)
  4. 4. Percent of businesses that fail Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Year 8 Year 9 Year 10 71%69%66%63%60%55%50%44%36%25% http://www.statisticbrain.com/startup-failure-by-industry/
  5. 5. Still operating after 4 years Finance Insurance and Real Estate Education and Health Agriculture Services Wholesale Mining Manufacturing Construction Retail Transportation Communication and Utilities Information 37% 45% 47% 47% 49% 51% 54% 55% 56% 56% 58% http://www.statisticbrain.com/startup-failure-by-industry/
  6. 6. Why firms die http://www.statisticbrain.com/startup-failure-by-industry/ Neglect, fraud, disaster 1% Lack of industry experience 13% Lack of managerial experience 34% Incompetence 52% Emotional Pricing Living too high for the business Nonpayment of taxes No knowledge of pricing Lack of planning No knowledge of financing No experience in record-keeping Poor credit granting practices Expansion too rapid Inadequate borrowing practices Carry inadequate inventory No knowledge of suppliers Wasted advertising budget
  7. 7. Part one:
 Introduction to Lean Analytics
  8. 8. Don’t sell what you can make. Make what you can sell. Kevin Costner is a lousy entrepreneur.
  9. 9. The core of Lean is iteration.
  10. 10. Waterfall approach You know the problem and the solution.
  11. 11. Known set of requirements Known ways to satisfy them Spec Build Test Launch
  12. 12. Agile methodologies Know the problem, find the solution
  13. 13. Known set of requirements Unclear how to satisfy them Build Test LaunchViable?Problem
 statement Adjust Sprints Unknown set of requirements
  14. 14. Lean approach First, know that you don’t know.
  15. 15. Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Product/market hypothesis Trialstartup Product/market hypothesis Trialstartup You are herePIVOT
  16. 16. Unfortunately, we’re all liars.
  17. 17. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) This is where things fall apart. No data, no learning.
  18. 18. Most startups don’t know what they’ll be when they grow up. Hotmail
 was a database company Flickr
 was going to be an MMO Twitter
 was a podcasting company Autodesk
 made desktop automation Paypal
 first built for Palmpilots Freshbooks
 was invoicing for a web design firm Wikipedia
 was to be written by experts only Mitel
 was a lawnmower company
  19. 19. Gottfried von Leibniz was a geek just like you.
  20. 20. First calculator (stepped reckoner) One of the first to recognize the importance of binary. “I thought again about my early plan of a new language or writing-system of reason, which could serve as a communication tool for all different nations..”
  21. 21. The best of all possible worlds is the one in which the fewest starting conditions produce the greatest variety of outcomes.
  22. 22. Analytics can help.
  23. 23. Analytics is the measurement of movement towards your business goals.
  24. 24. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  25. 25. Why now? First: High rate of change of digital technologies & channels.
  26. 26. Arbitron and radio data
  27. 27. Times a song in “heavy rotation” is played daily 0 15 30 2007 2012 266
  28. 28. Some fundamentals. Analytics, performance, aggregation, and the right metrics
  29. 29. We’re bad at data.
  30. 30. I have two kids. At least one of them is a girl.
  31. 31. What are the chances the other is a boy?
  32. 32. BB BG GB GG
  33. 33. 2 of 3 (66%) are boys. GB GG BG
  34. 34. Fundamental:
 What makes a good metric?
  35. 35. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior
 changing What will you do differently based on the results you collect?
  36. 36. The simplest rule bad
 metric. If a metric won’t change how you behave, it’s a h"p://www.flickr.com/photos/circasassy/7858155676/
  37. 37. Metrics help you know yourself. Acquisition Hybrid Loyalty 70%
 of retailers 20%
 of retailers 10%
 of retailers You are just like Customers that buy >1x in 90d Once 2-2.5
 per year >2.5
 per year Your customers will buy from you Then you are in this mode 1-15% 15-30% >30% Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size Focus on (Thanks to Kevin Hillstrom for this.)
  38. 38. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  39. 39. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  40. 40. Rumsfeld on Analytics (Or rather, Avinash Kaushik channeling Rumsfeld) Things we know don’t
 know we know Are facts which may be wrong and should be checked against data. we don’t
 know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t
 know Are exploration which is where unfair advantage and interesting epiphanies live.
  41. 41. MayAprMarFeb Slicing and dicing data Jan 0 5,000 Activeusers Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate
 analysis Changing several things at once to see which correlates with a result. ☀ ☁ ☀ ☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  42. 42. Which of these two companies is doing better?
  43. 43.   January February March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 Is this company growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April   $8 $7 May       $9 How about this one?
  44. 44. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1   March $7 $6 $5     April $8 $7       May $9         Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  45. 45. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  46. 46. Fundamental:
 Correlation
  47. 47. 1 10 100 1000 10000 Ice cream consumption Drownings Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
  48. 48. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  49. 49. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  50. 50. Is social action a leading indicator of donation? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
  51. 51. Is mobile use? http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/
  52. 52. Soundcloud Cory Levinson, Soundcloud
  53. 53. A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) Some examples (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  54. 54. AirBnB and Craigslist
  55. 55. http://www.flickr.com/photos/bootbearwdc/1243690099/ Think subversively.
  56. 56. Why is Nigerian spam so badly written?
  57. 57. Aunshul Rege of Rutgers University, USA in 2009 Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” 1000 emails 1-2 responses 1 fool and their money, parted. Bad language (0.1% conversion) Gullible (70% conversion) 1000 emails 100 responses 1 fool and their money, parted. Good language (10% conversion) Not-gullible (.07% conversion) This would be horribly inefficient since humans are involved.
  58. 58. Turns out the word “Nigeria” is the best way to identify promising prospects.
  59. 59. Nigerian spammers really understand their target market. They see past vanity metrics.
  60. 60. Fundamental: KISS
  61. 61. “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.” http://media.photobucket.com/image/einstein/derekabril/einstein_010.png
  62. 62. “As simple as possible,
 but no simpler.” *(FYI, this is irony.)
  63. 63. The Lean Analytics framework.
  64. 64. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  65. 65. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  66. 66. Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. Gate Thefivestages
  67. 67. Do you have a business model?
  68. 68. http://www.flickr.com/photos/tinfoilraccoon/197640807/ A lemonade stand’s business model is simple. Yours should be too.
  69. 69. A hobby. PhotobyRakkaonFlickr.https://www.flickr.com/photos/rakka/6889638004
  70. 70. Market Sizing Top-down and bottoms-up
  71. 71. Top-down Start with the whole planet, then narrow it down.
  72. 72. Population of Canada
  73. 73. Percent mobile phones by country
  74. 74. IOS penetration
  75. 75. Put this all together Metric/Assumption Value Population of Canada 34.88M Mobile phone percentage 80% Mobile phones in Canada 27,904M IOS percentage 15.5% IOS phones in Canada 4.32M IOS devices per account 3.1* IOS accounts we can sell to in Canada (this is our Total Addressable Market) 1.39M * I made this up. Don’t believe everything you read.
  76. 76. How much is the TAM worth? Metric/Assumption Value TAM 1.39M Percent we will claim 5% Number of users 69,500 User lifetime 40 months Revenue per month $5 per month CLV $200 Expected total revenues $13.9M * I made this up. Don’t believe everything you read.
  77. 77. Bottoms-up Start with one buyer, then add it up
  78. 78. Bottoms-up Metric/Assumption Value Impressions (acquisition) 9,266,667 Percent that install 10% Installs (activation) 926,667 Percent still using after 30 days 25% Become regular users (retention) 231,667 Percent that buy 30% Buy premium version (revenue) 69,500 Percent that churn each month 2.5% Customer lifespan 40 Revenue per month 5 Customer lifetime value 200 TAM value $13,900,000
  79. 79. Reality check Bottoms-up (TAM) Value Impressions (acquisition) 9.3 M impressions Top-down (TAM) Value IOS accounts we can sell to in Canada (this is our Total Addressable Market) 1.39 M users in Canada You Shall Not Pass.
  80. 80. Six business model archetypes. E-commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market The business you’re in
  81. 81. Can you move enough customers through this cycle to make more money than you spend? Make them refer others Retain them Get revenue from them Acquire customers Activate them http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
  82. 82. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Cancel Freemium churn Engaged User Free user disengagement Reactivate Cancel Trial abandonment rate Invite Others Paying Customer Reactivation
 rate Paid conversion FORMER USERS User Lifetime Value Reactivate FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Support data Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Capacity Limit Upselling rate Upselling Disengaged DissatisfiedTrial Over
  83. 83. Model + Stage = One Metric That Matters. One Metric
 That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG Empathy Stickiness Virality Revenue Scale Thestageyou’reat
  84. 84. Really? Just one?
  85. 85. Yes, one.
  86. 86. In a startup, focus is hard to achieve.
  87. 87. Having only one metric addresses this problem.
  88. 88. www.theeastsiderla.com
  89. 89. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  90. 90. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  91. 91. Empathy Stickiness Virality Revenue Scale E- commerce SaaS Media Mobile
 app User-gen
 content 2-sided
 market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals (Money from transactions) Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers (Money from active users) Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses (Money from ad clicks)
  92. 92. Better: bit.ly/BigLeanTable
  93. 93. Drawing some lines in the sand.
  94. 94. A company loses a quarter of its customers every year. Is this good or bad?
  95. 95. Not knowing what normal is makes you do stupid things.
  96. 96. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  97. 97. It’s oxygen You need customers to keep learning It’s a substitute for solvency PhotobyPaulMilleronFlickr.https://www.flickr.com/photos/94674772@N03/8788576498
  98. 98. Baseline: 10% visitor engagement/day Fred Wilson’s social ratios 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently
  99. 99. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  100. 100. Baseline: Calculating customer lifetime 25%
 monthly churn 100/25=4
 The average customer lasts 4 months 5%
 monthly churn 100/5=20
 The average customer lasts 20 months 2%
 monthly churn 100/2=50
 The average customer lasts 50 months
  101. 101. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV $200 CAC Now segment those users! 1/3 spend
  102. 102. Who is worth more? Today A Lifetime: $200 Roberto Medri, Etsy B Lifetime: $200 Visits
  103. 103. The Lean Analytics cycle
  104. 104. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  105. 105. The minimum needed to
 test the core idea Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient reads survey question Recipient responds to question Recipient installs mobile app Recipient creates account, profile Recipient sees survey results Recipient can edit profile, etc. 10-25%RESPONSERATE 70-90%RESPONSERATE
  106. 106. Do AirBnB hosts get more business if their property is professionally photographed?
  107. 107. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  108. 108. 5,000 shoots per month by February 2012
  109. 109. Hang on a second.
  110. 110. Gut instinct (hypothesis) Professional photography helps AirBnB’s business SRSLY?
  111. 111. Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a linePick a KPI
  112. 112. “Gee, those houses that do well look really nice.” Maybe it’s the camera. “Computer: What do all the highly rented houses have in common?” Camera model. With data:
 find a commonality Without data: make a good guess
  113. 113. Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  114. 114. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User interaction Customer satisfaction KPI dashboardsUser segmentation User analytics Spying on users
  115. 115. Part two:
 Why big companies
 need to change
  116. 116. (http://csinvesting.org/2012/01/06/fortune-500-extinction/) F500 Life Expectancy Growth by entering a new business 95
 % fail Corporate Strategy Board 99
 % fail Clay Christensen 75 years 15 years 1950 2010...
  117. 117. mikemace.com The slow death of a market leader. Revenue over time This is what most managers track. Note that sales keep rising (making you feel safe) until you run off the edge of the cliff. “Let’s cut prices to accelerate our growth.” “Time to enter the mainstream. Cut prices.” “We may miss the quarter. Let’s do a price promotion.” “That wasn’t supposed to happen. We’ll have to lay some people off. Gross margin percent Declining profit per unit (gross margin) is actually your best signal of trouble. The adoption curve Here’s where you actually are, but you don’t know it because you can’t draw the curve until after the market saturates. Early adopters Late adopters
  118. 118. In other words, if your job is change you have your work cut out for you.
  119. 119. Lesson one: Companies die because they fail to move to new business models.
  120. 120. Clay Christensen, The Innovator’s Dilemma CostperMB $1000 $100 $10 $1 Time 14” M ainfram e 8” M inicom puter 5.25” Desktop 3.5” N otebook
  121. 121. Technologies outstrip what the market needs, driven by feedback from the “best” current customer. Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time 8” 5.25” High end customer Low end customer
  122. 122. The new market has different criteria for success, which are uninteresting to incumbents. Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time Storage
 capacity Portability
  123. 123. Sometimes this has unintended consequences Clay Christensen, The Innovator’s Dilemma $1000 $100 $10 $1 Time Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity
  124. 124. Why now? First: High rate of change of digital technologies & channels.
  125. 125. Arbitron and radio data
  126. 126. Times a song in “heavy rotation” is played daily 2007 2012 266
  127. 127. This alone explains the collapse of modern print media. Circulation, annually Clicks, instantaneously Letters to the editor, weekly Hashtags, always
  128. 128. Why now? Second: It’s no longer about whether you can build it—it’s about whether anyone will care.
  129. 129. The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Martin Greenberger, ed., The Johns Hopkins Press, 1971.)Herbert Simon
  130. 130. Lit motors tests the risky part
  131. 131. Lesson two: The difference between a rogue agent and a special operative is permission.
  132. 132. If big firms can’t innovate, it’s this guy’s fault.
  133. 133. Technology has radically changed the incremental cost of businesses.
  134. 134. http://www.flickr.com/photos/ebolasmallpox/3733059220/ Software is eating the world.
  135. 135. An economic order quantity of one. Crafted Mass- produced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc... This is why software is eating the world.
  136. 136. Sustainable competitive advantage allows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage
  137. 137. Scale is now a liability. Compete on cycle time.
  138. 138. CW&T make a pen
  139. 139. http://www.flickr.com/photos/art_es_anna/288880795/
  140. 140. Optimizing the probable means discounting the possible.
  141. 141. This isn’t about a lack of resources.
  142. 142. http://www.flickr.com/photos/maladjusted/5207565912
  143. 143. Blockbuster had a lot going for it.
  144. 144. Plenty of inventory, of course. But that matters less than...
  145. 145. ...market intelligence, customers, existing payment approval, and customer history.
  146. 146. The problem was framing: Blockbuster thought it was in the video store management business. Netflix realized it was in the entertainment delivery business.
  147. 147. YOU ARE HERE
  148. 148. YOU ARE HERE LOCAL
 MAXIMUM OPTIMIZATION
 OF CURRENT METRICS
  149. 149. YOU ARE HERE GLOBAL
 MAXIMUMINNOVATION
 WITH NEW
 RULES
  150. 150. YOU ARE HERE SHORT-TERM INVESTORS
 HATE GOING DOWNHILL
  151. 151. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone First mover advantage happens long before the market emerges.
  152. 152. Constraints slow things down vs.
  153. 153. Capital cycles don’t fit the short, iterative nature of startup uncertainty 12 month budgeting cycle; annual plan. Future based on past. Agile, scrum, lean iterations. Today’s model. No evidence about the future. Project Project Project Project Project Project Project Project Project Project Project Project (Requires budget insulation)
  154. 154. What’s the biggest problem
 in a hair salon?
  155. 155. The empty chair. Knowing this, how might you change the client/customer relationship?
  156. 156. Can and must What triggers cultural change?
  157. 157. “Using Cue, you can tell if someone has the ‘flu in 10 minutes.” http://tiltthewindmill.com/when-can-becomes-must/
  158. 158. “Using Cue, you must tell if someone has the ‘flu in 10 minutes… …or Johnny can’t come on the school trip.”
  159. 159. Part three: Three maxima of innovation
  160. 160. When you’re a startup your goal is to find a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.
  161. 161. Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.
  162. 162. Business model vs. company stage Company size/age Early stage Big/incumbent B2B Target
 market B2C LessWoM
 Moreformaldecisions Slower cycle time
 More legacy constraints It is way too easy to mix these up. Intrapreneurs
  163. 163. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  164. 164. In a big company, analytics replaces opinion with fact.
  165. 165. Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and profits than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011). 2011 MIT study of 179 large publicly traded firms
  166. 166. The fundamental shift from Big Data Ask question Define
 schema Collect
 data Answer
 question Refine problem Collect data Ask question Emergent schema Explore data Answer
 question “Collect first; ask questions later.”
  167. 167. Three kinds of innovation Sustain/core
 (optimizing for more of the same) Innovate/adjacent
 (introduce nearby product,
 market, or method) Disrupt/transformative
 (Fundamentally changing
 the business model) Improve along current metrics... ...or alter
 the rate of improvement Switch to a new value model Change the business
 model entirely
  168. 168. Many models for enterprise innovation Core Adjacent Transformative Do the same thing better. Nearby product, market, or method. Start something entirely new. Regional
 optimizations. Innovation, go-to- market strategies. Reinvent the business model. • Get there faster • Smaller batches • Solution, then testing • Increased accountability • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins
  169. 169. Another way to look at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall:
 Execution matters Agile/scrum:
 Iteration matters Lean Startup: Discovery matters
  170. 170. Experiment with product, market, and method.
  171. 171. Product (new “what”) Market (new “who”) Method (new “how”) 3 kinds of innovation
  172. 172. Engine as a service
  173. 173. http://www.nasa.gov/images/content/365835main_airplane_noise_qtd2_3024x2016.jpg Engine as a service
  174. 174. “Efficiency is tied to analytics. We’ll still look for new materials, or for the physics of devices, but the analytics ... is what’s really untapped.”
  175. 175. Current
 state Business optimization (five mores)
 Product,
 market,
 method innovation
 Business model innovation
 You can convince executives of this because some of it is familiar. This terrifies them because it eats the current business. A three-maxima model of enterprise innovation
  176. 176. Improvement Adjacency Remodeling Do the same,
 only better. Explore what’s
 nearby quickly Try out new
 business models Lean approaches apply, but the metrics vary widely. Sustain/
 core Innovate/
 adjacent Disrupt/
 transformative
  177. 177. Sustaining Adjacent Disruptive Next year’s car Electric car,
 same dealer On-demand, app-based
 car service
  178. 178. Sustaining innovation is about more of the same. (says Sergio Zyman) More things To more people For more money More often More efficiently Supply chain optimization Per-transaction cost reduction Loyal customer base that returns Demand prediction, notification Maximum shopping cart Price skimming/tiering Highly viral offering Low incremental order costs Inventory increase Gifting, wish lists
  179. 179. Blizzard extends the lifespan of WOW Early
 adopters Rapid
 growth Market
 saturation The infamous S-curve (Product lifecycle, Bass diffusion curve, etc.)
  180. 180. Blizzard extends the lifespan of WOW
  181. 181. Blizzard extends the lifespan of WOW Fixing this: sustaining growth with novelty Product & market innovation (“New & improved!”)
  182. 182. Blizzard extends the lifespan of WOW WOW Burning
 Crusade Wrath of
 the Lich King Mists of
 PandariaCataclysm Warlords of Draenor
  183. 183. Most of your innovation will be adjacent or sustaining. Question marks! (low market share, high growth rate) May be the next big thing. Consumes investment, but will require money to increase market share. Stars! (high growth rate, high market share) What everyone wants. As market invariably stops growing, should become cash cows. Dogs! (low market share, low growth rate) Barely breaks even, may be a distraction from better opportunities. Sell off or shut down. Cash cows! (high market share, low growth rate) Boring sources of cash, to be milked but not worth additional investment. Growthrate Market share Pivot to 
 increase
 market
 share
 through
 virality,
 attention Pivot to
 increase growth
 rate through
 disruption Pivot to
 redefine problem/
 solution through
 empathy Milk with
 revenue
 optimization as
 growth slows If you don’t like this, go launch a startup.
  184. 184. Software, experimentation, and iterative cycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.
  185. 185. Adjacent innovation is about changing one part of the model in a way that alters the value network.
  186. 186. Amazon Web Services and the server value network Server computing • Density • Heat • GHz • MIPS Cloud computing • Instances • Objects • Spinup time • Scaleout Capex, financing, TCO, ROI Opex, demand, time to result CIO, enterprise IT CTO, coder, app owner, line of business, startup Value
 criteria Money Buyer
  187. 187. Adjacent product to the same market in the same way
  188. 188. Selling the same product to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps
  189. 189. Selling the same product to the same market in a new way. The biggest innovation in logistics of the 20th century. http://www.flickr.com/photos/photohome_uk/1494590209
  190. 190. Changing the method of C2C classifieds A blend of who, what, how Classified C2C sales (same “what”) Strictly for Japanese women (targeted “how”) New how (phone is capture, display, payment, transaction) Did 100 interviews w/target users before launch Key insight: Japanese women sell their entire wardrobe twice in their lives 5,000 and 10,000 sales in first month 10% commission fee Average price of items is pretty low, at around 2,000 to 3,000 yen (or $22 to $34) Not an auction: seller decides price Mobile-only model Phone is payment, storefront, and even a way for sellers to build their catalog http://www.sffashtech.com/2012/10/10/a-free-market-fashion-app-exclusively-for-women-japan/
  191. 191. Selling the same product to the same market in a new way.
  192. 192. (At this point, observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)
  193. 193. Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.
  194. 194. If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.
  195. 195. Transformative isolation: Skunkworks
  196. 196. Transformative incubation: Metlife Infinity
  197. 197. Significant market 850K full-time law enforcement officers in the US; 700K state/local; 525K patrol officers 130M incident reports/y. 70M new incidents; 200K involve use of force Only 31% of local police agencies keep computer files on use-of-force incidents Strong product benefits Exonerates the officer 96% of the time. 47% percent increase in charges and summons (2007) Patrol officers spend 15-25% of their time writing incident reports, recorded evidence reduces this by 22%, meaning 50m more on patrol Challenges New business model Pricing unclear SaaS offering Compliance and governance Unions, regulation, chain of evidence Changing the current model (radio is everything) Transformative incubation: Taser evidence.com
  198. 198. Part four: What works for large companies (A bagful of tricks from agitators in companies of all sizes.)
  199. 199. The job of an intrapreneur is to identify an adjacent market, product, or method that conforms to organizational filters. It is not to improve the current product, market, or method.
  200. 200. Also: a pariah. Successful innovators share certain attributes. Bad listener: Wilfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.
  201. 201. Know what kind of innovation you’re after. New Current Current New Market Product Penetrate:
 Increase revenues, market share, product quality, brand differentiation. Marketing. Market development:
 Sell existing products to new markets, segments, uses. Export & license. Product development:
 Invent new products for your market. R&D, enhancements. Acquisition. Startup:
 New products for new markets. New rules, business units, organizational structure. Innovation. Based on H. Igor Ansoff’s matrix Increased risk of political fallout (and great success!)
  202. 202. Innovation portfolios at big companies Core Adjacent Transformative 70% 20% 10% Investment 70%20%10% Return
  203. 203. Use outliers and missed searches to hunt for good ideas & adjacencies (Multi-billion-dollar hygiene product company) 1/8 men have an incontinence issue. 1/3 women do. When search results show a significant number of men searching, this suggests the adjacent (male) market is underserved.
  204. 204. Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139
  205. 205. When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.
  206. 206. Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.
  207. 207. 4” e-ink display with name and specialty. Badge scans barcode and gets specs; checks inventory; enters data on a touch screen. Smart Badge Today: Workers see their own productivity. Coming soon: comparing yourself to 400,000 other employees. Ultimately: Learning what (and who) works well. Data Exhaust Tesco connects its workforce
  208. 208. Don’t just collect data, chase it.
  209. 209. Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.
  210. 210. Know what has to be built in-house SAP integration Employment law
  211. 211. http://www.flickr.com/photos/bootbearwdc/1243690099/ Think subversively.
  212. 212. Everything’s an excuse to experiment
  213. 213. Find other ways to collect data; everything is an experiment.
  214. 214. Run it as a consulting business first. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and find the 20% of features that 80% of the market will pay for.)
  215. 215. Convince your boss she asked for this Draw a new line Pivot or
 give up Try again Success! Did we move the needle? Measure the results Make changes in production Design a test Hypothesis With data:
 find a commonality Without data: make a good guess Find a potential improvement Draw a line
 in the sand Pick a KPI
  216. 216. Focus on the desired behavior, not just the information. http://www.psychologytoday.com/blog/yes/ 200808/changing-minds-and-changing-towels 26% increase in towel re-use with an appeal to social norms; 33% increase when tied to the specific room. Energy Conservation “Nudges” and Environmentalist Ideology: Evidence from a Randomized Residential Electricity Field Experiment - Costa & Kahn 2011 The effectiveness of energy conservation “nudges” depends on an individual’s political ideology ... Conservatives who learn that their consumption is less than their neighbors’ “boomerang” whereas liberals reduce their consumption.
  217. 217. Take baby steps.
  218. 218. Netflix
  219. 219. Tesla http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
  220. 220. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  221. 221. Figure out how to translate it back to a simple model that fits the company’s existing value model. If your company dies, this is why.
  222. 222. Software Platform Merchandising User-generated content Marketplace Media/content Service Oracle’s accounting suite Amazon’s EC2 cloud Thinkgeek’s retail store Facebook’s status update AirBnB’s list of house rentals CNN’s news page A hairstylist Product
 type What the startup does in return. May be a product or service; may be hardware or software; may be a mixture. One-time transaction Recurring subscription Consumption charges Advertising clicks Re-sale of user data Donation Single purchase from Fab Monthly charge from Freshbooks Compute cycles from Rackspace PPC revenue on CNET.com Twitter’s firehose license Wikipedia’s annual campaign Revenue
 model How the startup extracts money from its visitors, users, or customers. Paid advertising Search Engine Mgmt. Social media outreach Inherent virality Artificial virality Affiliate marketing Public relations App/ecosystem mkt. Banner on Informationweek.com High pagerank for ELC in kid’s toys Active on Twitter i.e. Kissmetrics Inviting team member to Asana Rewarding Dropbox user for others’ signups Sharing a % of sales with a referring blogger Speaker submission to SXSW Placement in the Android market Acquisition
 channel How the visitor, customer, or user finds out about the startup. Hosted service Digital delivery Physical delivery Salesforce.com’s CRM Valve purchase of desktop game Knife shipped from Sur La Table Delivery
 model How the product gets to the customer. Simple purchase Discounts & incentives Free trial Freemium Pay for privacy Free-to-play Buying a PC on Dell.com Black Friday discount, loss leader, free ship Time-limited trial such as fitbit Premium Free tier, relying on upgrades, like Evernote
 Free account content is public, like Slideshare Monetize in-app purchases, like Airmech Selling
 tactic What the startup does to convince the visitor or user to become a paying customer.
  223. 223. Acquisition
 channel Selling
 tactic Revenue
 model Product
 type Delivery
 model Business aspect Flipbook
 page(s) Inherent virality. Artificial virality. Sharing files with others. Free storage when others sign up. Freemium. Limited-capacity accounts are free; subscribe when you need more. Recurring subscription. $99/year, monthly fees, enterprise tiers. Platform. Storage-as-a-service with APIs, collaboration, synchronization tools. Hosted service. Digital delivery. Cloud storage, web interface. Desktop client software. Dropbox example
  224. 224. Use the right proxies for B2B products Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Stickiness Churn, engagement Support tickets, integration time, call center data, delays Virality Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing
  225. 225. Tomorrow’s company: Running parallel businesses Innovation Sustaining/core Adjacent Transformative/ disruptive Core action Optimizing/ improving Experimenting Searching/ inventing Focus on Known metrics Risk removal Assumption validation Which will live Within current business unit Incubated, then integrated As new/separate entities Problem is Known Known Unknown Solution is Known Unknown Unknown
  226. 226. How to build a message map
  227. 227. I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  228. 228. People who want to drive Prospective car buyers People looking for a hybrid Honda Civic Hybrid owners I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  229. 229. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model- specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  230. 230. People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  231. 231. People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. Financing, cashback Sell to carshares; underscore their limitations PR on dangers of commuting, pedestrian deaths Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  232. 232. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model- specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. Financing, cashback Sell to carshares; underscore their limitations PR on dangers of commuting, pedestrian deaths Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) I need a carA. I should buy
 a car B. It should be
 a hybridC. I should buy
 a Honda CivicD. Everyone in the world
  233. 233. Hits A metric from the early, foolish days of the Web. Count people instead. Page views Marginally better than hits. Unless you’re displaying ad inventory, count people. Visits Is this one person visiting a hundred times, or are a hundred people visiting once? Fail. Unique visitors This tells you nothing about what they did, why they stuck around, or if they left. Followers/friends/ likes Count actions instead. Find out how many followers will do your bidding. Time on site, or pages/visit Poor version of engagement. Lots of time spent on support pages is actually a bad sign. Emails collected How many recipients will act on what’s in them? Number of downloads Outside app stores, downloads alone don’t lead to lifetime value. Measure activations/active accounts.
  234. 234. Example: a restaurant • Empathy: Before opening, the owner first learns about the diners in its area, their desires, what foods aren’t available, and trends in eating. • Stickiness: Then he develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. He’s giving things away, testing things, asking diners what they think. Costs are high because of variance and uncertain inventory. • Virality: He starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. He engages on Yelp and Foursquare. • Revenue: With virality kicked off, he works on margins—fewer free meals, tighter controls on costs, more standardization. • Scale: Finally, knowing he can run a profitable business, he pours some of the revenues into marketing and promotion. He reaches out to food reviewers, travel magazines, and radio stations. He launches a second restaurant, or a franchise based on the initial one.
  235. 235. Example: a software company • Empathy: The founder finds an unmet need, often because she has a background in a particular industry or has worked with existing solutions that are being disrupted. • Stickiness: She meets with an initial group of prospects, and signs contracts that look more like consulting agreements, which she uses to build an initial product. She’s careful not to commit to exclusivity, and tries to steer customers towards standardized solutions, charging heavily for custom features. She supports the customers directly from the engineering team until the product is stable and usable. • Virality: Product in hand, she asks for references from satisfied customers, and uses them as testimonials. She starts direct sales, and grows the customer base. She launches a user group, and starts to automate support. She releases an API, encouraging third-party development and scaling potential market size without direct development. • Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored based on anticipated payoff and development cost. Recurring license and support revenue becomes an increasingly large component of overall revenues. • Scale: She signs deals with large distributors, and works with global consulting firms to have them deploy and integrate her tool. She attends trade shows to collect leads, carefully measuring cost of acquisition against close rate and lead value.
  236. 236. Empathy Get inside their head.
  237. 237. How to avoid leading the witness Don’t tip your hand Avoid biased wording, preconceptions, or a giveaway appearance. Word your surveys carefully to be neutral. Make the question real Get them to purchase. Ask them to pay. Demand real introductions. Or ask them “how many of your friends would say X” to avoid self-effacement Keep digging Ask “why” several times. Leave lingering, uncomfortable pauses in the conversation and let them fill them. Look for other clues Have a colleague make notes of when they react, or of their body language.
  238. 238. Mobile app model:
 Localmind hacks Twitter • Stage: Empathy • Model: UGC/mobile • Real-time question and answer platform tied to locations. • Needed to find out if a core behavior—answering questions about a place— happened enough to make the business real
  239. 239. Localmind hacks Twitter • Before writing a line of code, Localmind was concerned that people would never answer questions. • This was their biggest risk: if questions went unanswered users would have a terrible experience and stop using Localmind. • Ran an experiment on Twitter • Tracked geolocated tweets in Times Square • Sent @ messages to people who had just tweeted, asking questions about the area: how busy is it; is the subway running on time; is something open; etc. • The response rate to their tweeted questions was very high. • Good enough proxy to de-risk the solution, and convince the team and investors that it was worth building Localmind.
  240. 240. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place Ask what existing brands come to mind in an industry Market alongside them? Address competitors? Choose partners? Ask how customers try to find a product or service Help you plan marketing campaigns and choice of media Ask what kind of money people spend on a problem Shape your pricing strategy Test which tagline or unique value proposition resonates best with customers Choose the winning one, or just take that as advice
  241. 241. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey Demographic segmentation questions Quantifiable answers to your research problem Qualitative, open- ended feedback
  242. 242. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes)
  243. 243. • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes) • Send it out Audience plea Creating an answers-at-scale campaign Via your NW To a paid list As an ad campaign Beware of respondent bias, misrepresentation of the larger market Beware of spamminess, low open rates Name the problem Give the solution or unique value “Are you a single mom? Take this brief survey and help us address a big challenge.” “Can’t sleep? We’re trying to fix that, and want your input.”) “Our accounting software automatically finds tax breaks. Help us plan the product roadmap.”
  244. 244. Creating an answers-at-scale campaign • Know why you’re doing a survey in the first place • Design the survey • Test it (you’ll always have mistakes) • Send it out • Collect the results • Analyze the data Were you able to capture the attention of the market? Did they click on your ads and links? Which ones worked best? Are you on the right track? What decisions can you now make with the data you’ve collected? Will people try out your solution/product? How many of your respondents were willing to be contacted? How many agreed to join a forum or a beta? How many asked for access in their open-ended responses? By
 segment!
  245. 245. When it’s time to move on • Have you conducted enough quality customer interviews to feel confident that I’ve found a problem worth solving? • Do you understand your customer well enough? • Do you believe your solution will meet the needs of customers?
  246. 246. Stickiness The dogs like the dogfood.
  247. 247. Hits1995 Visits1997 Visitors1999 Conversions2002 Engagement2010 Who did you add? Where from? Why? What did they do? How did it benefit? Who did you lose? Why did they leave?
  248. 248. Days since last visit 1200 1000 800 600 400 200 0 1 2 3 4 5 6 7 8 9 Days since last engagement January February Disengaged (>10 days) 25000 20000 15000 10000 5000 0 Numberofusers
  249. 249. When it’s time to move on • Are people using the product as expected? • Define an active user. What percentage of your users/customers is active? Write this down. Could this be higher? What can you do to improve engagement? • Evaluate your feature roadmap against the 7 questions to ask before building more features. Does this change the priorities of feature development? • Evaluate the complaints you’re getting from users. How does this impact feature development going forward?
  250. 250. Virality I told two friends.
  251. 251. ------------------------------------------------------
 Get your free private email at http://www.hotmail.com
 ------------------------------------------------------ Viral coefficient
  252. 252. v ≠ 1, pt  = δp0 (1 – vt+1) / (1 – v) + p0  http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ Viral coefficient
  253. 253. Or simpler Users Viral
 coefficient Churn &
 abandonment x - > 1
  254. 254. How to calculate it • First calculate the invitation rate, which is the number of invites sent divided by the number of users you have. • Then calculate the acceptance rate, which is the number of signups or enrollments divided by the number of invites. • Then multiply the two together. • Consider, for example • Your 2,000 customers have sent out 5,000 invitations during their lifetime on your site. • Your invitation rate is 2.5. • For every ten invitations received, one gets clicked. • Your acceptance rate is 0.1. • Multiply the two, and you have your viral coefficient: 0.25. Every customer you add will add an addition 25% of a customer.
  255. 255. Virality stage:
 Timehop’s content sharing • Stage: Virality • Model: Mobile app • Social network around the past • Focused on virality (but not necessarily the coefficient!)
  256. 256. The one metric that matters: content sharing • Focused on percent of daily active users that share their content • Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else—is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
  257. 257. 3 kinds of virality • Inherent virality is built into the product, and happens as a function of use. • Artificial virality is forced, and often built into a reward system. • Word of mouth virality is simply conversations generated by satisfied users.
  258. 258. Laid, paid, made, afraid Who is your audience? Drive-by visitors vs. serious evaluators; loyal users vs. one-time buyers; lurkers vs. contributors. Have one audience. What do you want them to do? Tell five friends; sign up for a year’s subscription; write a blog post on your behalf; invest some money. Just be clear. Why should they do it? Drive-by visitors versus serious evaluators; loyal returning users versus one-time buyers; lurkers versus contributors Power or respect among their peers; appeal to reputation. Money, or alternate currency rewards; an appeal to greed or compensation. Sex, attractiveness; an appeal to desire. Fear of missing out, risk, loss, etc.; an appeal to safety. Made Paid Laid Afraid Humans have four big motivators:
  259. 259. When it’s time to move on • Are you using one of the three types of virality (inherent, artificial, word of mouth) for your startup? Describe how. If virality is a weak aspect of your startup, write down 3-5 ideas for how you could build more virality into your product. • What’s your viral coefficient? Even if it’s below 1 (which it likely is), do you feel like the virality that exists is good enough to help sustain growth and lower customer acquisition costs? • What’s your viral cycle time? How could you speed it up?
  260. 260. Revenue Pour some of the money back into acquisition.
  261. 261. What’s a fitbit customer worth? • The user can record their steps with a device in their pocket • They can use it and sync data to the hosted application • They can visit the portal to see their statistics • They can manually enter sleep and food data • They can buy the premium Fitbit offering • Each of these is a different tier of engagement, and Fitbit could segment users across these five segments when analyzing the effectiveness of a marketing campaign or the volume of support e-mails.
  262. 262. When it’s time to move on You’re making money You’re sustainable You’re tracking growth metrics
  263. 263. Revenue stage:
 Backupify’s customer lifecycle • Stage: Revenue • Model: SaaS • Leading backup provider for cloud based data. • The company was founded in 2008 by Robert May and Vik Chadha • Has gone on to raise $19.5M in several rounds of financing.
  264. 264. You need a business model Start with a systems diagram What’s the core job to be done?
  265. 265. Start with the customer journey.
  266. 266. Consider the morning routine. Wake up What happens here? Start workday
  267. 267. What job are you doing? Personal dimension Social dimension Functional aspects Emotional aspects Related jobs to be done Functional aspects Emotional aspects Personal dimension Social dimension Main job to be done http://innovatorstoolkit.com/content/technique-1-jobs-be-done
  268. 268. Changes create new JTBD answers Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953 + = ?
  269. 269. Understand culture What will it take for a change to really stick?
  270. 270. Whither
 Estonia?
  271. 271. How about now?
  272. 272. Some stats • 215M digital signatures • 18 minutes to start a company, entirely online • 1/3 of voting happens via the Internet
  273. 273. Get rid of the pens:
 Digital signatures since 2000.
  274. 274. Only collect things once:
 Tax forms 95% completed already.
  275. 275. Conclusions
  276. 276. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  277. 277. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  278. 278. ARCHIMEDES HAD TAKEN BATHS BEFORE.
  279. 279. Once, a leader convinced others in the absence of data.
  280. 280. Now, a leader knows what questions to ask.
  281. 281. Alistair Croll acroll@gmail.com @acroll Ben Yoskovitz byosko@gmail.com @byosko
  282. 282. Day two business models
  283. 283. Hardcore interviews
  284. 284. Maybe they don’t love you
 like they said they do. N Your offering doesn’t make them want to
 brag or their contact isn’t really a friend N Advocates can’t learn & convey
 your message easily N They don’t trust you entirely N Woohoo! Scalable, viral,
 explainable product! Y Get a meeting Y Grab the phone Y They pitch it Y Call them now? Y Intro to a friend? Interview
  285. 285. B2B and the 80/20 rule How to tell if you have a sustainable, repeatable B2B idea.
  286. 286. Target customer Reachable, referenceable. Find 10 prospects* Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. Target customer Reachable, referenceable. * 10 is an aribtrary number. Just be sure there are enough out there to grow sustainably from their actions.
  287. 287. Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardize 80% of the offering
  288. 288. Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Standardized (sameprocesses,docs,tools) Carefully learn what’s custom (& costly)Custom Custom Custom Custom Custom Custom Custom Custom Custom Custom
  289. 289. Standardize, automate, patentCustom Custom Custom Custom Custom Custom Custom Custom Custom Custom Your margins, differentiation, and scalable growth come from this. Without it, you’re either a consultant; or the market isn’t ready; or the tech isn’t possible yet. And that’s OK because you know.
  290. 290. The whole point of digital is personal Segment 1 User segment
 (who) Segment 2 Segment 3 Goal
 (what) Goal 1 Goal 2 Goal 3 Motivation
 (why) Goal 1 Goal 2 Goal 3
  291. 291. http://www.thirdwunder.com/funnels-are-a-horrible-metaphor/Photo Credit: Patrick McGarvey More like this.
  292. 292. The leaky bucket of subscriptions. Photo by Jeffrey on Flickr - https://www.flickr.com/photos/jb912/7479464134
  293. 293. Bounces Non-
 creators Non-
 payers Churn Signup
 rate Engagement
 level Conversion
 rate Visitors Subscribers Engaged users Paying
 customers Leaky buckets are a bad metaphor
  294. 294. Bounces Non-
 creators Non-
 payers Churn Improve stickiness; call-to-action optimization;
 A/B page testing; picking better traffic sources Notifications; updates; reactivating users;
 segmenting those who are engaged. Usage caps, natural upselling;
 premium features Better support; credit card renewal; naturally sustained features
  295. 295. Let them ask for an out Detecting SaaS churn early without hurting cashflow
  296. 296. The tradeoff Charge a monthly fee Charge annual fee up front Find out if they hate it sooner, when they cancel on first billing cycle. No need to pay back CAC; cash you can use right away. Takes months to recoup the money you spent acquiring them. May be a zombie user who vanishes when the year is up.
  297. 297. They’re happy, you keep your money, goodwill for offering. They’re unsatisfied, you made it right, they tell you why, you learn. The solution—because the goal is to learn, not to trap customers. Annual fee, with an out 1. Offer an annual, discounted fee. 2. After a couple of weeks, ask if they’d like a refund.
  298. 298. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU App sales Activation Churn, CLV In-app" purchases Appstore! Incentivized Legitimate Fraudulent Ratings!
  299. 299. Targeting matters.
  300. 300. It’s all about cabbages
  301. 301. Start with the customer journey.
  302. 302. Consider the morning routine. Wake up What happens here? Start workday
  303. 303. What job are you doing? Personal dimension Social dimension Functional aspects Emotional aspects Related jobs to be done Functional aspects Emotional aspects Personal dimension Social dimension Main job to be done http://innovatorstoolkit.com/content/technique-1-jobs-be-done
  304. 304. Changes create new JTBD answers Photo by Andrew Dyer on Flickr - https://www.flickr.com/photos/andrewdyer/350938953 + = ?
  305. 305. I lied. Everyone is a tech company.
  306. 306. http://www.flickr.com/photos/puuikibeach/4789015423 http://www.flickr.com/photos/elcapitanbsc/3936927326 Cost of experiments: down. Cost of attention: way up.
  307. 307. Let’s pick on restaurants for a while.
  308. 308. A line in the sand Labor costs Gross revenue 30% 24% 20% Too costly? Just right Understaffed? =
  309. 309. A leading indicator http://www.flickr.com/photos/avlxyz/4889656453http://www.flickr.com/photos/mysticcountry/3567440970 50 reservations
 at 5PM 250 covers
 that night (Varies by restaurant. McDonalds ≠ Fat Duck.)
  310. 310. http://www.flickr.com/photos/southbeachcars/6892880699 Restaurant MVP
  311. 311. Is tip amount a leading indicator of long- term revenue?
  312. 312. Why does every table get the same menu?
  313. 313. Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf

×