Lean analytics for startups - Leweb2010

Lean analytics for startups

Wednesday, December 8, 2010

@acroll
(if you have questions)


Lean Not lean


Ries, Mclure, and Blank
are often misquoted.


They never said “fail faster.”

Instead:
“Learn and adapt.”


Waterfall, agile, and lean
Three approaches for three situations.


Waterfall approach
You know the problem and the solution.


Known ways to Known set of
satisfy them requirements

Spec Build Test Launch


Agile methodologies
Know the problem, ﬁnd the solution


Unclear how Unknown set
Known set of
to satisfy them of requirements
requirements

Problem
statement Build Test Viable? Launch

Sprints

Adjust


Most startups
Don’t even know what problem they solve.


Possible viable
offering

IVOT You are
P

Trial startup
here

Possible Possible Possible
viable Trial startup problem Trial startup viable
offering space offering
Trial startup

Possible viable
offering

All startups are lean at ﬁrst.
Which has some interesting consequences.


Knowing
Getting what’s not
features right working,
matters less. quickly,
matters more.


“We should” “What if”
matters less. matters more.


Your Parallel
roadmap experiments
matters less. matter more.


Real-world
User groups adoption &
and surveys usage
matter less. analysis
matters more.


Business Business
plans matter models matter
less. more.


http://www.ﬂickr.com/photos/tinfoilraccoon/197640807/

Revenue ( Lemonade ) x Units sold + tips

Cost ( Lemon + cups + sugar ) x Units made
( Markers + cardboard ) x Signs made
( Stafﬁng ) x Hours worked

Proﬁt!

(maybe not that fun a game)

http://www.ﬂickr.com/photos/ethanhein/3207831470/#/

Clouds and analytics
It’s never been a better time to start a company.


Clouds make Analytics
experimenting make
easy, avoid experimenting
upfront smart & fast.
dilution.


I love to talk
about clouds
(but not today)

http://www.ﬂickr.com/photos/ecastro/3053916892/

I’m going to focus on the
other part
What to watch on your website, if you’re a startup.


So what could we watch?


“Hard” data

Analytics Usability Performability
(what did they (how did they (could they do
do on the interact with what they
site?) it?) wanted to?)

Complete Web Monitoring
Community VoC Competition
(what were (what were (what are they
they saying?) their up to?)
motivations?)

“Soft” data

Analytics is the
measurement of movement
towards your business
goals.

http://www.ﬂickr.com/photos/itsgreg/446061432/


ATTENTION ENGAGEMENT CONVERSION

NEW
VISITORS
SEARCHES GROWTH CONVERSION
PAGES TIME RATE
TWEETS NUMBER
OF VISITS
PER ON x
MENTIONS VISIT SITE
GOAL
ADS SEEN LOSS
VALUE
BOUNCE
RATE


Visits

Shopping cart

Payment options

Conversions


KPIs
Bounce

rate
Conversion
Visits rate

Shopping cart

Payment options

Conversions


http://www.ﬂickr.com/photos/mrmoorey/160654236


http://www.ﬂickr.com/photos/duncan/1252272164/

http://www.ﬂickr.com/photos/intherough/3573333256/

http://www.ﬂickr.com/photos/jetheriot/648950773/

Impact of page load time on
average daily searches per user
0%

-0.15%

-0.30%

-0.45%

-0.60%
50ms pre-header 100ms pre-header 200ms post-header 200ms post-ad 400ms post-header


Impact of additional delay on
business metrics
0%

-1.25%

-2.50%

-3.75%

-5.00%

50 200 500 1000 2000
Queries/visitor Query reﬁnement Revenue/visitor
Any clicks Satisfaction

Shopzilla’s makeover


5-12%

Trafﬁc levels
9,000
Total number of visits

6,750

4,500
8,505

2,250 4,740

0
Optimized Unoptimized
Visitor experience

Bounce rate
20

15
Visits that bounced

10

13.38% 14.35%
5

0
Visitor experience

Average time on site
31
Time on site (minutes)

23

16 30.17
23.83
8

0
Visitor experience

Pages per visit
16
Average pages seen

12

8 15.64
11.04
4

0
Visitor experience

Conversion rate
20
and order value
Diﬀerence due to optimization

15

10
16.07
5

5.51
0
Conversion rate Order value


Community management

Hugging
Bunnies
Retweets
Friends!


Unpaid search Community Email Banner ad
mentions campaign

?
• Google PageRank • Open rate • Cost of ads
• Sessions-to-clicks • Opt-out rate (CPM)
ratio • Clickthrough
rate
Visits

Shopping cart

Payment options

Conversions

But there’s a danger:
vanity metrics.


So what should you watch?
The lean analytics shortlist.


If you remember only one thing...

In a startup, the purpose of
analytics is to iterate to a
product/market ﬁt before
the money runs out.


Viral coefﬁcient
Message reach Ampliﬁcation

Infrastructure health Performability

Customer research
Market sentiment Community sentiment

User engagement Time since last visit

Core goals

Business model Extended funnel abandonment
Movement towards goals

Cost per engaged visitor

Sustainability Peak to average ratio

Minimum sustainable burn

Viral coefﬁcient
On average, for each message they hear, how
many people does someone tell?

1

------------------------------------------------------
Get your free private email at http://www.hotmail.com
------------------------------------------------------


v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0

http://robert.zubek.net/blog/2008/01/30/viral-coefﬁcient-calculation/

How well are messages
being ampliﬁed?

2

Downstream reach
(aggregate followers)

50% 75% 100% 50% =4

2 3 1 =6
Ampliﬁcation chance
(how relevant the content 25% 50% 50%
is + what the recipient
thinks of the sender)

.5 2 1 = 3.5
14.5


Seed (starting
community)
Repurposing
Reach
(spread to other
(impressions)
communities)
Visits

Shopping cart Ampliﬁcation
(virality and
Payment options message spread)

Conversions


Viral message spread

Reach
(impressions) Emphasis on
Visits getting viral
ratio above 1
Shopping cart (Retweeting,
Fan, Email
Payment options forward, Reddit
upvote, other
Conversions loops)


Megablogger proponents
Seed (starting
Emphasis on
community)
convincing highly-
Reach followed, highly
(impressions)
acted-upon seed
Visits group to spread
the word.
Shopping cart

Payment options

Conversions


A call to action

Reach
(impressions)
Visits

Shopping cart Emphasis on
maximizing
Payment options impression-to-
click ratio within
Conversions the community


Goal attainment: funnel
Seed ratio:
20
35,000:1

Followers 700,000s 2 Repurposing

150 Ampliﬁcation:
Visitors: 2,000 RT 2.9%
1,642
0.23%

7 x $100 Revenues:
Conversions:
1.95% 32 10 x $20 Average: $39.54
Median: $20
15 x $7
Total: $1,005

How performable are we?
Health of the infrastructure (uptime, latency)

3

5,000 $10,000

Revenue (total sales)
Unique page views

Conversions

>4s or error

2-4s

0 $0 <2s

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16


What do visitors think of us?
Both on the site (VoC) and in the world
(community monitoring)

4&5

How engaged are users?
Time since last visit, as a histogram, not an
average, compared to baselines.

6

Days since last visit

1200 25000
Number of users

January February
1000
20000
800
15000
600
10000
400
5000
200

0 0
1 2 3 4 5 6 7 8 9 Disengaged
Days since last engagement (>10 days)


Core goals driving the
business
Pick the 3-4 tasks you’ve set for users. Always
know what these are and why they correlate to
business growth.

7

(Remember the model?)
Revenue ( Lemonade ) x Units sold + tips

Cost ( Lemon + cups + sugar ) x Units made
( Markers + cardboard ) x Signs made
( Stafﬁng ) x Hours worked

Proﬁt!

Extended funnel
abandonment
Not just on the site: Email open rate

8

In the last build, what
moved us towards & away
from goals?
Requires knowing the goals and the key features
added

9

Cost per visitor
Operations costs per site visitor

10

(this had better be greater
than the revenues and
conversion rate.)


Peak-to-average ratio
How much busier are you at busy times?
What premium do you pay for that surplus?
Can you decommission well?

11

http://community.microsoftadvertising.com


12

http://www.ﬂickr.com/photos/dougww/833127501/

Viral coefﬁcient
Message reach Ampliﬁcation

Infrastructure health Performability

Market sentiment
Customer research
Community sentiment 12
User engagement Time since last visit things
Core goals to
Business model
watch
Extended funnel abandonment
Movement towards goals

Cost per engaged visitor

Sustainability Peak to average ratio


Thanks!
alistair@bitcurrent.com
@acroll
www.yearonelabs.com
www.rpmmtl.com
www.watchingwebsites.com
www.bitcurrent.com
www.human20.com


Lean analytics for startups - Leweb2010

More Related Content

Viewers also liked

Similar to Lean analytics for startups - Leweb2010

More from Alistair Croll

Recently uploaded

Lean analytics for startups - Leweb2010