An introduction to how deploying experimentation at speed and scale leads to Customer Experience and Business Model success.

1. BUSINESS
EXPERIMENTATION
An introduction to how deploying experimentation at speed and
scale leads to Customer Experience and Business Model success.
THANK YOU
HELGE TENNØ

2. ENTREPRENEURS WHO ACT LIKE
SCIENTISTS PERFORM BETTER
Figure 1: Average
revenue over time
(euros), treated and
control startups
Pivoting to a greater extent increasing their odds
of pursuing projects with greater returns.
The treatment consists of training the treated
group to identify the problem, articulate
theories, define clear hypotheses, conduct
rigorous tests to prove or disprove them,
measure the results of the tests, and make
decisions based on these tools.
Source: A SCIENTIFIC APPROACH TO ENTREPRENEURIAL DECISION-MAKING:
EVIDENCE FROM A RANDOMIZED CONTROL TRIAL [with 116 startups]
Arnaldo Camuffo, Alessandro Cordova and Alfonso Gambardella
Figure 2: Content
of training steps

3. THREE TYPES OF HYPOTHESES:
PROBLEM-
TO-SOLVE
CUSTOMER NEED /
MOTIVATION
CUSTOMER VALUE
PROPOSITION
What is the problem we
think is worth solving?
What progress is the
customer trying to achieve
or which struggle are they
trying to overcome within
the context of the problem
What value can we offer the
customer to that would lead
them to the progress they are
trying to achieve and deliver
business value to us?
“75% of startups fail
because they are trying to
solve problems that are not
really problems”
- Remko Vermeulen
PROBLEM CUSTOMER SOLUTION

4. PROBLEM HYPOTHESIS:
What is the problem we
think is worth solving?
WHAT IS OUR PROBLEM-TO-SOLVE?
“Let’s find a problem for
this solution”
- Marc Andreessen -
Andreessen Horowitz
Problem statements are simple statements highlighting what we want
to achieve, with whom, where/when and with which intended outcome?
will be able to better understand,
experiment with and manage their
own energy consumption.
What is the intended outcome?
(For our company and the customer)
We believe that by
Direction of
improvement
(More or less of
something)
increasing
Verb
(What is the ‘type’
of improvement)
the insights
in their own
detailed
energy use
What and / or who is the verb
referring to
(e.g. what or who is being affected)
the small business
owner and
operations
manager
del
365
STRUCTURE:
EXAMPLE:
We believe that Direction of
improvement
(More or less of
something)
Verb
(What is the ‘type’
of improvement)
What and / or who is
the verb referring to
(e.g. what or who is being
affected)
What is the intended outcome?
(For our company and the customer)
Suggested resources:
PROBLEM-
TO-SOLVE

5. What progress is the
customer trying to achieve
or which struggle are they
trying to overcome within
the context of the problem
CUSTOMER HYPOTHESIS:
WHAT IS THE CUSTOMER NEED?
To define the customer need we use Customer Jobs-to-be-done
identifying the customers underlying need and using their own voice.
towards a more
environmentally
sustainable future
Contextual clarifier
As a small
business owner
I want to
Direction of improvement
(More or less of something)
do more, smarter
and better
Verb
(What is the ‘type’
of improvement)
measures to
increase
What and / or who is the verb
referring to
(e.g. what or who is being affected)
my operational
performance
del
365
STRUCTURE:
EXAMPLE:
What progress
Is the customer
trying to achieve
Which
situation
Suggested resources:
OR
What struggle
Are they trying to
overcome
IN AND
How to
measure
In the
customers
own voice:
Clayton Christensen on Jobs-
to-be-done and milkshake
Clayton Christensen on
knowing your customers jobs-
to-be-done
Anthony Ulwick on Giving your
customers a fair hearing
Anthony Ulwick on Outcome
Driven Innovation, Jobs-to-be-
done theory in practice
CUSTOMER
JOBS-TO-BE-
DONE

6. What value can we offer the
customer to that would lead
them to the progress they are
trying to achieve and deliver
business value to us?
SOLUTION HYPOTHESIS:
WHAT VALUE CAN WE OFFER THE CUSTOMER?
A Customer Value Proposition articulates the company’s offer of value to the
customer. We use it to frame our value offering before we build solutions.
STRUCTURE:
CUSTOMER VALUE
PROPOSITION
Suggested resources:
Can you say what
your strategy is?
What is a Value
Proposition?
The Customer Value
Proposition is a part
of the Business
Model Canvas
Strategyzer’s Value
Proposition Canvas
Which combination
or sum of our own
capabilities and
activities
By Leading us to
offer which
value for the
customer
Supporting the
customers job
in what way
Within which scope
(for the customer)
Delivering to what
measurable object
(to the customer)
By
Which combination or sum of our
own capabilities and activities
combining our
innovative business
model with our
sophisticated
technology
del
365
EXAMPLE:
we can offer the customer
a tangible / manageable
understanding of their
operations energy
consumption
supporting them
towards targeted
and specific
measures (actions)
delivering to
their own
sustainable
future
Leading us to offer which value
for the customer
Supporting the customers
job in what way
Within which scope
(for the customer)
Delivering to what
measurable object
(to the customer)

7. ASSUMPTIONS
RISKIEST ASSUMPTIONS TESTING #1
“A riskiest assumptions test puts the focus on learning .. that allows us to move forward one step at a time. Once you’ve validated
the riskiest assumption you can move on to the next largest one. Gradually building confidence in the viability of your idea”
- Rik Higham
“Maximizing the rate of learning by
minimizing the time to try things”
- Tom Chi, co-founder of Google X
Breaking your hypothesis down to smaller bets (assumptions) allows you to
co-create with your customers increasing your speed and volume of learning.
STEP 1 - IDENTIFY ALL YOUR ASSUMPTIONS:
Simply ask: “what has to be true for our hypothesis to be true?”
A. Collect all assumptions as post-its. B. Cluster and remove duplicates
Riskiest assumptions testing
moves your learning from being
extrinsic to your customer
interactions to being a part of
them. You co-create your
solutions with the customer in
real-time.
Suggested resources:
The MVP is dead,
long live the RAT
SUPPORT:
EXAMPLE (HYPOTHESIS JOBS-STATEMENT):
1. Go through the hypothesis almost word for word and
ask: “What has to be true for this to be true?” e.g. of a
person says they need a drink, they must be thirsty.. or
are there any other reason for wanting to drink?
2.Look at the statement from a broader lens. What is it
you know about the world that would have implications
on this statement? e.g. legal, technology, barriers etc.
Think bigger or think sideways.
3.Looking at this statement, what is it you want to learn
more about? e.g. if the statement is regarding the
drinking (mentioned in bullet point 1) are they talking
about water, wine or whiskey?
As a small business owner I want to do more, smarter and better measures
to increase my operational performance towards a more environmentally
sustainable future
del
365
ASSUMPTIONS:
“There is not a lack of
ideas, but ideas
management”
- Florian Hunger-Reichardt -
1. I am motivated by a more environmentally sustainable future
2. I am committed to perform operational measures
3. I think there is a lot more to learn (I don’ t know everything)
What you don’t know
you don’t know

8. ASSUMPTIONS
RISKIEST ASSUMPTIONS TESTING #2 - PRIORITIZATION
Having identified all your assumptions you now might have 5, 15, 50 of more.
This is good. But now you need to prioritize them so you know where to start.
STEP 2 - PRIORITIZATION
Riskiest assumptions testing
moves your learning from being
extrinsic to your customer
interactions to being a part of
them. You co-create your
solutions with the customer in
real-time.
Suggested resources:
The MVP is dead,
long live the RAT
“There is not a lack of
ideas, but ideas
management”
- Florian Hunger-Reichardt -
“Data precedes framework” - Dave Snowden
There is no correct way to priortize assumptions. I personally see the best result when the team just starts organizing the
assumptions from left to right or top to bottom. As some consensus and logic starts to emerge the team discusses why
they find one assumption to be more important than the other and from that discussion a framework starts to emerge.
Please remember: When scoring an assumption
you are creating an assumption
about an assumption
The prioritization is just there to support
your teams discussion on where to
start. The team can choose any
assumption to do next no matter the
prioritization
Your list of assumptions are a living
document. Anyone, anywhere can at
any time suggest new assumptions and
ad them to the prioritization.
EXAMPLE:
del
365
As a small business
owner I am
motivated by a more
environmentally
sustainable future
As a small business
owner I am
committed to
perform operational
measures
As a small business
owner I think there
is a lot more to
learn (I don’ t
know everything)
More importantLess important
CUSTOMER
JOB-
STATEMENT:

9. EXPERIMENT
TURNING YOUR ASSUMPTION INTO EXPERIMENTS
An experiment is defined by a clear and simple statement: if we do a then b
happens. The statement needs to be testable and the outcome measurable.
What do you want to learn?
Articulate the assumption using the
experiment statement:
HOW TO ARTICULATE YOUR EXPERIMENT:
How will you measure the
outcome?
Where do you want to run your
experiment(s)?
If we offer an option of
different value propositions
a significant amount would
choose the one about
environment sustainability
Measure in terms of click-
through / engagement
a) Website
b) Search Advertising
c) LinkedIn-advertising
As a small business
owner I am
motivated by a more
environmentally
sustainable future
EXAMPLE:
del
365
“If we do a then b will happen” Using what metric / data
In order to deliver the necessary
accuracy / commitment

10. EXPERIMENT
CO-CREATE YOUR ASSETS AND ENVIRONMENT
With your whole team aligned around a shared understanding
of your experiment co-create the tasks and responsibilities
to build the necessary assets and environments.
Meet up in an
online collaborative
environment.
e.g. miro.com or
mural.co
STEPS TO CO-CREATE:
Everyone is aware of
the experiment, now
ask everyone to
individually note down
every task they know
needs to be done to
build, review, deploy
and run the assets and
environment.
Everyone is give 4-5 minutes
to add their tasks
Cluster the tasks
into logical
categories.
e.g. creative, design,
content, development, etc.
Add names to each
category
Everyone individually adds
their own name or the name
of others they know needs
to contribute
Add tasks and
responsibilities to
sprint backlog
Or other preferred tool
depending on the teams own
way of working
a) b) c) d) e)

11. STRUCTURE
& RIGOR
PROBLEM
CUSTOMER
SOLUTION
RISKIEST
ASSUMPTIONS
TESTING
COST OF
COMMITMENT
SPEED
VOLUME
BUSINESS EXPERIMENTATION
SUMMARY
CO-CREATE
COORDINATE
INSIGTHS-
TO-ACTION
THANK YOU
HELGE TENNØ