The proof that lean startup saves time and money Dr. Max Völkel, 2016

1. Lean Startup
Cost and Benefits
The proof that lean startup
saves time and money
Dr. Max Völkel, 2016

2. COPYRIGHTDR.MAXVÖLKEL.,2015
Lean Startup Method
■ You make a plan
■ Be honest, your plan is just a set of guesses
and hopes  we call them assumptions
■ Try to validate your assumptions by running
cheap experiments, which collect hard data.
■ So you build experiments, measure the
outcome, and learn, how you need to adapt
the plan (“build-measure-learn”)
2

3. COPYRIGHTDR.MAXVÖLKEL.,2015
Assumptions A
■ A business model can be represented as 9
building blocks in the Business Model
Canvas (BMC), from Alex Osterwalder. 
9 assumptions
■ Add some assumptions about competition,
your team, your sales numbers, market
size and you’ll easily have 30-50
assumptions.
3

4. COPYRIGHTDR.MAXVÖLKEL.,2015
Probability p(A)
■ An assumption might be
true, or it might not.
Let’s say an
assumption A is true
with probability p(A).
4
p(A)
100%
50%
0%
If we have no information
whether A is true or not,
p(A)=50% (fifty-fifty).
If p(A) is < 50%, we have the
weird situation that our plan
is based on an assumption,
which we ourselves already
think is rather not true.
If p(A)=70%, there is a 30% risk
of A being false.

5. COPYRIGHTDR.MAXVÖLKEL.,2015
Cost(A)
■ If A turns out to be wrong, you need to “adapt your plan”. That can
be finding another customer segment, choosing a new (hopefully
better) channel, tweaking the product, … in all cases, you need to
do extra work.
■ Extra work for “fixing” your risky project has a cost.
■ Cost can be quantified in Dollars or Euros, or in person-work-days.
Choose a unit and stick with it. Let’s measure in work days. And we
really mean time spent working, not waiting.
■ So every failed assumptions has a cost(A), which is really the cost
that happens only if A is not true.
5

6. COPYRIGHTDR.MAXVÖLKEL.,2015
Ecost(A)
■ Now we turn to statistics. We need a concept
called expected value.
 Example: The expected value of a dice showing
numbers 1,2,3,4,5,6 is 3.5.
■ Ecost(A) = (1-p(A)) * Cost
 If an assumption A has p(A)=50% and cost(A)=80
days, Ecost(A)=40 days.
• If we would do the same risky project again and again, on average we would spend 40 days
doing extra work due to failed hypotheses. Of course, in reality, we only do a startup once, but
as we have 30 or more assumptions, they average out and we will spend, on average the
expected value of cost.
■ Ecost is really just the expected value of cost.
6

7. COST AND BENEFIT
WITHOUT LEAN STARTUP
7

8. COPYRIGHTDR.MAXVÖLKEL.,2015
Without Lean Startup
■ We have 30 Assumptions
 Yes, could be any number, but a range 30-50 is
common
■ We estimated for each assumption the probability
of being true, and the cost of extra work, if the
assumption is wrong.
■ During out project, some assumptions will turn out
to be wrong. We just don’t know which.
■ At the end of the project, we will have spent
approximately the sum of all Ecost values for
doing extra work.
8

9. COPYRIGHTDR.MAXVÖLKEL.,2015
Example with only 5 Assumptions
Assumption A p(A) Cost(A)
in days
1-p(A) Ecost(A)
=(1-
p(A))*Cost
Assumption 1 50% 20 0,50 10,00
Assumption 2 70% 5 0,30 1,50
Assumption 3 66% 5 0,33 1,65
Assumption 4 80% 1 0,20 0,20
Assumption 5 50% 8 0,50 4,00
Sum 39 17,35
9
Everything else: CalculatedGuessed

10. COPYRIGHTDR.MAXVÖLKEL.,2015
Example with only 5 Assumptions
10
Assumption A p(A)
Assumption 1 50%
Assumption 2 70%
Assumption 3 66%
Assumption 4 80%
Assumption 5 50%
■ If all assumptions are right,
nothing goes wrong, we have 0
extra days of work – very
unlikely.
• Probability of nothing going wrong for our example is
0,5*0,7*0,66*0,8*0,5 = 9,24%. This number if much smaller for
more assumptions.
■ If absolutely everything goes
wrong (Murphys Law), we have
39 days of extra work – very
unlikely.
■ We should expect extra work of
17,35 days = sum of Ecost
values.

11. COST AND BENEFIT
WITH LEAN STARTUP
11

12. COPYRIGHTDR.MAXVÖLKEL.,2015
Experiments are costly, too
■ Right, so let’s call doing experiments validation and let
Cost_V be the cost of doing an experiment (planning,
doing, evaluating data, learning).
■ The whole purpose of the experiment is to move our
vague estimated probability p(A) closer to 100%.
■ Let’s accept that we don’t always achieve that.
 Maybe we had p(A)=60% and after talking to 100 potential
customer we now believe A is true with p = 90%.
■ Let’s call the probability after the validation p_V(A).
 Note that even though p_V(A) > p(A), the cost(A) remains
the same. If things go wrong, we need to do the same
amount of work, no matter how sure we have been, that a
particular assumption would be true.
12

13. COPYRIGHTDR.MAXVÖLKEL.,2015
Is the experiment worth it?
■ The expected cost Ecost_V(A) after running an
experiment for validating assumption A is
Ecost_V(A) = (1-p_V(A)) * Cost + Cost_V
 Note how we add to cost of running the experiment
(Cost_V) to the expected costs.
■ The benefit of running the experiment is
Ecost(A) - Ecost_V(A)
 On average, we should spend less time on extra work
now, because we have a plan that is based more on
true facts and less on wishful thinking or naïve
assumptions. Of course, if an experiment shows an
assumption is wrong, we remove the assumption from
our plan and adapt the plan.
13

14. COPYRIGHTDR.MAXVÖLKEL.,2015
Test every assumption?
14
Assumption A p(A) Cost(A)
in days
1-p(A) Ecost(A) Cost_V p_V(A) Ecost_V(A)
Assumption 1 50% 20 0,50 10,00 4 95% (1-0,95)*20+4=5,0
Assumption 2 70% 5 0,30 1,50 1 80% (1-0,80)*5+1=2,0
Assumption 3 66% 5 0,33 1,65 6 80% (1-0,80)*5+6=7,0
Assumption 4 80% 1 0,20 0,20 2 90% (1-0,9)*1+2=2,1
Assumption 5 50% 8 0,50 4,00 2 90% (1-0,9)*8+2=2,8
Sum 39 17,35 18,90
Everything else: CalculatedGuessedGuessed

15. COPYRIGHTDR.MAXVÖLKEL.,2015
A Shock!
■ As you saw on the last slide, the expected
cost without doing any experiments was
17,35 days, and with doing the best
experiment we could imagine for each
assumption, our sum of expected costs is
18,9 days. Higher! So Lean Startup is just
causing extra work?
■ We should certainly not run all possible
experiments.
■ Can we do better?
15

16. COPYRIGHTDR.MAXVÖLKEL.,2015
Do only some Experiments
Assumption A p(A) Cost(A)
in days
1-p(A) Ecost(A) Cost_V p_V(A) Ecost_V(A)
Assumption 1 50% 20 0,50 10,00 4 95% (1-0,95)*20+4=5,0
Assumption 2 70% 5 0,30 1,50 1 80% (1-0,80)*5+1=2,0
Assumption 3 66% 5 0,33 1,65 6 80% (1-0,80)*5+6=7,0
Assumption 4 80% 1 0,20 0,20 2 90% (1-0,9)*1+2=2,1
Assumption 5 50% 8 0,50 4,00 2 90% (1-0,9)*8+2=2,8
■ For A2, A3 and A4 we should not run the experiments we
came up with.
■ Only our experiments for A1 and A5 improve the expected
costs. In total, we expect 7,8 days of work instead of 14. So
we gain 6,2 days, just for being smart.

17. COPYRIGHTDR.MAXVÖLKEL.,2015
Summary
■ Lean Startup is about doing experiments
which reduce your total expected cost.
■ You invest extra work (the experiments) and
in return you get either
 a) a plan based on true assumptions, with data to
proof it (which in turn can convince partners and
investors) – or –
 b) a new plan, because you found out early
where your plan was based on false assumptions
before you went ahead and spent a lot of time
and money building the product.
17

18. COPYRIGHTDR.MAXVÖLKEL.,2015
Thank You
■ Version 1 published on 2016-02-15
■ Copyright 2016 by Max Völkel,
www.xam.de
■ Licensed under Creative Commons CC
BY-NC-SA 3.0, see
https://creativecommons.org/licenses/by-
nc-sa/3.0/
18