19. WHAT TO A/B TEST
Short Answer: Everything
Longer Answer: That you have
enough traffic to see significant
results that are not trival
optimisations (HINT you are not
Bing/Google)
20. WHAT TO A/B TEST
Every new feature should go
into A/B testing.
If has negative or no real effect,
pull it.
Optimise existing features/copy/
design (80% existing 20% new)
32. T MY DR ILL
R EN
Marketplace to rent/buy drills from
your neighbour (AirBnB for drills)
Takes a small percentage of the
transaction between two parties
33. T MY DR ILL
R EN
Key business drivers are:
Two sided business model
means acquisition of both
lenders and borrowers and
transactions per user
35. ONE KEY METRIC
There is generally only one key
metric you need to concentrate
on at a time. Such as referral or
sign-up conversion or
cancellation rate.
39. MEASURING COST OF VISITS
Cost per click (site visit)
Cost of campaign / traffic
generated
(Social media) Cost of effort /
traffic generated
40. MEASURING COST OF
ACQUISITION
Cost of one visit / conversion
rate = Cost of an acquisition
Acquisition could be paid for
acquisition or free sign-up
Unfortunately not that simple
42. MARKETING CHANNELS
Per channel track:
Scale
Cost
Conversion
Optimise and scale channels
with best Cost of Acquisition
(CoA) and potential to Scale
43. ACQUISITION
Cost of Acquisition < Customer
Lifetime Profit
Then scale acquisition at a
similar or lower rate
A/B test acquisition channels to
optimise
44. ACQUISITION INCLUDING REFERRAL
If it costs £3 to acquire a
customer through advertising
But that customer brought in 2
other customers through referral
The COA of that channel is £1
47. ACTIVATION AT FACEBOOK
(i) which data points predict
whether a user will stay?
(ii) if they stay, which data points
predict how active they’ll be
after three months?
48. ACTIVATION AT FACEBOOK
(i) having more than one session as
a new user, and entering basic
profile information.
(ii) how often a user was reached
out to by others, frequency of third
party application use, and how
forthcoming a user was on the site.
51. RETURNING USERS
Tracking unregistered users that
return often (useful for content
sites with advertising)
Tracking registered users that
return and take some sort of
action.
57. VIRAL COEFFICIENT - EXAMPLE
Each Doggiecollr customer
invites 5 other customers by
email
20 % click on that email link
50 % of those users convert into
customers
58. VIRAL COEFFICIENT - EXAMPLE
Viral coefficient =
Refferals x click through x conversion
5 x 0.2 x 0.5 = 0.5
59. VIRAL COEFFICIENT
A viral co-effiecient greater than
1 means that every customer
gets more than one other
customer on average
Therefore your product will
grow virally
61. VIRAL CYCLE TIME
The average time taken for a
referral to turn into a customer
With the viral coeffiecent and
viral cycle time you can
determine your viral growth
62. NET PROMOTER SCORE
Number of people who would
recommend your product to a
friend.
A single question survey
63. NET PROMOTER SCORE
How likely is it you would
recommend AgileSamba to a
friend or colleague?
0 1 2 3 4 5 6 7 8 9 10
NPS = % Promoters - % Detractors
64. NET PROMOTER SCORE
Survey a small percentage of
your customers by email weekly
Record who you sent to so you
don’t repeat the same people
every week
65. NET PROMOTER SCORE
NPS is not a direct
measurement that translates to
key metrics
Can act as a leading indicator of
other metrics
67. MAKING MONEY
Key for any business is to find
out how much profit they make
for every customer and scaling
the number of customers
68. LIFETIME CUSTOMER VALUE
How much money you make for
every customer you acquire
For some this is a single
transaction
For subscription businesses it
depends on retention
69. CANCELLATION RATE
Is the percentage number of
customers who cancel in any
given month compared to total
(paying) customers
[Cancellation rate] = [product
utility] + [service quality] +
[acceptable price]
70. LIFETIME CUSTOMER VALUE (LTV)
LTV = monthly revenue x no
months in lifetime
No months = 1 / cancellation
rate
LTV = monthly revenue /
cancellation rate
71. LIFETIME CUSTOMER VALUE (LTV)
For example:
If £20 a month and cancelation
rate is 10% a month
LTV = £200
72. LIFETIME CUSTOMER VALUE (LTV)
Often not that simple. Short term
cancellation rates are much
higher than long term
Use Cohort Analysis to
determine
73. LIFETIME CUSTOMER VALUE (LTV)
r = short-term cancellation rate
(e.g. 0.15)
p = long-term cancellation rate
(e.g. 0.03)
s = number of months in the
“short-term” age group (e.g. 3)
(1-r)^s × (s + 1/p) = expected
months