Cost and Benefits
The proof that lean startup
saves time and money
Dr. Max Völkel, 2016
Lean Startup Method
■ You make a plan
■ Be honest, your plan is just a set of guesses
and hopes we call them assumptions
■ Try to validate your assumptions by running
cheap experiments, which collect hard data.
■ So you build experiments, measure the
outcome, and learn, how you need to adapt
the plan (“build-measure-learn”)
■ A business model can be represented as 9
building blocks in the Business Model
Canvas (BMC), from Alex Osterwalder.
■ Add some assumptions about competition,
your team, your sales numbers, market
size and you’ll easily have 30-50
■ An assumption might be
true, or it might not.
Let’s say an
assumption A is true
with probability p(A).
If we have no information
whether A is true or not,
If p(A) is < 50%, we have the
weird situation that our plan
is based on an assumption,
which we ourselves already
think is rather not true.
If p(A)=70%, there is a 30% risk
of A being false.
■ If A turns out to be wrong, you need to “adapt your plan”. That can
be finding another customer segment, choosing a new (hopefully
better) channel, tweaking the product, … in all cases, you need to
do extra work.
■ Extra work for “fixing” your risky project has a cost.
■ Cost can be quantified in Dollars or Euros, or in person-work-days.
Choose a unit and stick with it. Let’s measure in work days. And we
really mean time spent working, not waiting.
■ So every failed assumptions has a cost(A), which is really the cost
that happens only if A is not true.
■ Now we turn to statistics. We need a concept
called expected value.
Example: The expected value of a dice showing
numbers 1,2,3,4,5,6 is 3.5.
■ Ecost(A) = (1-p(A)) * Cost
If an assumption A has p(A)=50% and cost(A)=80
days, Ecost(A)=40 days.
• If we would do the same risky project again and again, on average we would spend 40 days
doing extra work due to failed hypotheses. Of course, in reality, we only do a startup once, but
as we have 30 or more assumptions, they average out and we will spend, on average the
expected value of cost.
■ Ecost is really just the expected value of cost.
Without Lean Startup
■ We have 30 Assumptions
Yes, could be any number, but a range 30-50 is
■ We estimated for each assumption the probability
of being true, and the cost of extra work, if the
assumption is wrong.
■ During out project, some assumptions will turn out
to be wrong. We just don’t know which.
■ At the end of the project, we will have spent
approximately the sum of all Ecost values for
doing extra work.
Example with only 5 Assumptions
Assumption A p(A) Cost(A)
Assumption 1 50% 20 0,50 10,00
Assumption 2 70% 5 0,30 1,50
Assumption 3 66% 5 0,33 1,65
Assumption 4 80% 1 0,20 0,20
Assumption 5 50% 8 0,50 4,00
Sum 39 17,35
Everything else: CalculatedGuessed
Example with only 5 Assumptions
Assumption A p(A)
Assumption 1 50%
Assumption 2 70%
Assumption 3 66%
Assumption 4 80%
Assumption 5 50%
■ If all assumptions are right,
nothing goes wrong, we have 0
extra days of work – very
• Probability of nothing going wrong for our example is
0,5*0,7*0,66*0,8*0,5 = 9,24%. This number if much smaller for
■ If absolutely everything goes
wrong (Murphys Law), we have
39 days of extra work – very
■ We should expect extra work of
17,35 days = sum of Ecost
Experiments are costly, too
■ Right, so let’s call doing experiments validation and let
Cost_V be the cost of doing an experiment (planning,
doing, evaluating data, learning).
■ The whole purpose of the experiment is to move our
vague estimated probability p(A) closer to 100%.
■ Let’s accept that we don’t always achieve that.
Maybe we had p(A)=60% and after talking to 100 potential
customer we now believe A is true with p = 90%.
■ Let’s call the probability after the validation p_V(A).
Note that even though p_V(A) > p(A), the cost(A) remains
the same. If things go wrong, we need to do the same
amount of work, no matter how sure we have been, that a
particular assumption would be true.
Is the experiment worth it?
■ The expected cost Ecost_V(A) after running an
experiment for validating assumption A is
Ecost_V(A) = (1-p_V(A)) * Cost + Cost_V
Note how we add to cost of running the experiment
(Cost_V) to the expected costs.
■ The benefit of running the experiment is
Ecost(A) - Ecost_V(A)
On average, we should spend less time on extra work
now, because we have a plan that is based more on
true facts and less on wishful thinking or naïve
assumptions. Of course, if an experiment shows an
assumption is wrong, we remove the assumption from
our plan and adapt the plan.
■ As you saw on the last slide, the expected
cost without doing any experiments was
17,35 days, and with doing the best
experiment we could imagine for each
assumption, our sum of expected costs is
18,9 days. Higher! So Lean Startup is just
causing extra work?
■ We should certainly not run all possible
■ Can we do better?
Do only some Experiments
Assumption A p(A) Cost(A)
1-p(A) Ecost(A) Cost_V p_V(A) Ecost_V(A)
Assumption 1 50% 20 0,50 10,00 4 95% (1-0,95)*20+4=5,0
Assumption 2 70% 5 0,30 1,50 1 80% (1-0,80)*5+1=2,0
Assumption 3 66% 5 0,33 1,65 6 80% (1-0,80)*5+6=7,0
Assumption 4 80% 1 0,20 0,20 2 90% (1-0,9)*1+2=2,1
Assumption 5 50% 8 0,50 4,00 2 90% (1-0,9)*8+2=2,8
■ For A2, A3 and A4 we should not run the experiments we
came up with.
■ Only our experiments for A1 and A5 improve the expected
costs. In total, we expect 7,8 days of work instead of 14. So
we gain 6,2 days, just for being smart.
■ Lean Startup is about doing experiments
which reduce your total expected cost.
■ You invest extra work (the experiments) and
in return you get either
a) a plan based on true assumptions, with data to
proof it (which in turn can convince partners and
investors) – or –
b) a new plan, because you found out early
where your plan was based on false assumptions
before you went ahead and spent a lot of time
and money building the product.
■ Version 1 published on 2016-02-15
■ Copyright 2016 by Max Völkel,
■ Licensed under Creative Commons CC
BY-NC-SA 3.0, see