Hypotheses: Supercharging your ideas and improving your decision making (NUX Manchester)

Hypotheses
Supercharging your ideas and improving
your decision making
Tom Adams @sonofswiss
NUX, 9th April 2018

@sonofswiss
Think of an idea
you’ve had recently.

@sonofswiss
1.Ideas
2.Hypotheses
3.Better hypotheses
4.Using hypotheses to
prioritise and persuade

@sonofswiss
Cambridge Dictionary
https://dictionary.cambridge.org/dictionary/english/hypothesis
idea
noun /aɪˈdɪə/
a suggestion or plan for doing something.

“Make a lamb
and chickpea
stew with the
leftover roast
lamb from
Easter Sunday
https://www.jamieoliver.com/recipes/lamb-
recipes/lamb-chickpea-curry/
”

@sonofswiss
There is nothing inherent in an
idea that says whether it is a
good idea or a bad idea

@sonofswiss
belief
concept
hunch
intention
notion
opinion
perception
suggestion
Idea =

@sonofswiss
‘Experimenting’ isn’t
throwing any old idea at a
wall to see if it sticks.
It is our responsibility to
spend our budgets wisely.

@sonofswiss
We use bad ideas all
the time.

@sonofswiss
■ We are overconfident (or
underconfident) in our own ideas
Decisions get made too quickly
Decisions are often based around
emotion, politics, relationships
and unconscious biases - not
evidence

Hippos are dangerous.
Hippos are dumb.

Hippos are dangerous.
Hippos are dumb.
There are hippos making bad
decisions on your projects.

ghest
aid
erson’s
pinion
Hi
P
P
O

@sonofswiss
How can we tell if an idea is
good or bad, so we make
good decisions around it?

@sonofswiss
A hypothesis contains an
idea, plus extra information
about that idea.

@sonofswiss
– Ben Holliday, Design Director @ FutureGov
“Hypothesis driven design
helps teams take calculated risks
to move a product forward and
explore different solutions”

@sonofswiss
1. Come up with an idea
Clearly state what you think the idea will
achieve, and who for
Clearly state how you’ll measure
whether it actually achieves it

@sonofswiss
Jeff Gothelf, Josh Seiden, Lean UX
We believe this
[business outcome]
will be achieved if
[these users]
successfully
[attain this user outcome]
with [this feature]

@sonofswiss
We believe [idea]
For [users]
Will result in [outcome]
We’ll know this is true if [evidence]

@sonofswiss
■ What’s the idea you’re proposing?
■ Should be directional and non-ambiguous
■ May be a fully-formed idea, may be the
starting point of a design process
We believe [idea]

@sonofswiss
■ Which users* does this idea target?
(If you don’t understand your different user
types, stop everything and do that first)
*
For [users]

@sonofswiss
■ This is the expected outcome.
■ It should state a clear value to users
and/or the business
■ Should tie into strategic goals of the
product

@sonofswiss
■ What metric(s) can we measure
to show success?
■ Should be a quantitative measure
(one exception: if the product/feature is a prototype)
■ Consider measuring for unintended
consequences (defensive metrics)

@sonofswiss
Hypotheses have
saved the world!

“Make a lamb
and chickpea
stew with the
leftover roast
lamb from
yesterday
”

I believe
[making lamb and
chickpea stew] for
[us and the kids]
will result in [a
great dinner time
and no waste]. We’ll
know this is true if
[everyone eats and
enjoys their dinner]

I believe
[making lamb and
chickpea stew] for
[us and the kids]
will result in [a
great dinner time
and no waste]. We’ll
know this is true if
[everyone eats and
Text
=

“Make a lamb
and chickpea
stew with the
leftover roast
lamb from
yesterday
”
=

@sonofswiss
Conversion Optimisation / CRO

@sonofswiss
A hunch you can
measure is still a hunch.

@sonofswiss
We weren’t doing
enough user research.
(?)

@sonofswiss
1. There was little client buy-in for UR
When we did do UR, there was no clear
record of which UR fed into which
AB test
Also, the UR we were doing wasn’t
being used effectively in
prioritisation and decision making

@sonofswiss
1. There was little client buy-in for UR
When we did do UR, there was no clear
record of which UR fed into which
AB test
The UR we were doing wasn’t being
used effectively in prioritisation and
decision making

We believe [idea]
For [users]

@sonofswiss
How do we build an
evidence base into the
hypothesis, and use it to
persuade our HiPPOS?

@sonofswiss
3.Better hypotheses

@sonofswiss
Cambridge Dictionary
https://dictionary.cambridge.org/dictionary/english/hypothesis
hypothesis
noun / haɪˈpɒθ.ə.sɪs/ plural hypotheses
an idea or explanation for something
that is based on known facts but has
not yet been proved.

@sonofswiss
Craig Sullivan, Simple hypothesis kit
https://medium.com/@optimiseordie/hypothesis-kit-2-eff0446e09fc
1. Because we saw
(data/feedback)
2. We expect that (change)
will cause (impact)
3. We’ll measure this using
(data metric)

@sonofswiss
But isn’t this just
obvious?

@sonofswiss
Confidence not evidence
“How confident are we that
this idea will work?”

@sonofswiss
Because [things we know or believe]
We believe [idea]
For [users]

@sonofswiss
■ Be specific
■ Aim for a range of evidence

@sonofswiss
■ Be specific
■ Aim for a range of evidence
■ Don’t discount hunches!

Because [evidence?],
I believe
[making lamb and
chickpea stew] for
[us and the kids] will
result in [a great
dinner time and no
waste]. We’ll know
this is true if
[everyone eats and

@sonofswiss
User 1 - age 40 (F)
■ Never been a fussy eater
■ Generally clear about what food she
likes and doesn’t like
■ Likes lamb, chickpeas, couscous
■ Has previously eaten and enjoyed
similar dinner
■ Said “that sounds nice” to recipe idea

@sonofswiss
User 2 - age 2½ (M)
■ Very fussy eater, changes mind about
what he likes meal-to-meal and in-meal
■ Can’t communicate well what he likes
and doesn’t like
■ Loves couscous
■ Cried last time we gave him chickpeas
■ When eating roast lamb the day before,
said “I like lamb”, “I don’t like lamb”, “I
like lamb” again, then only ate one piece
of lamb

@sonofswiss
User 3 - age 5 (F)
■ Unpredictable eater, sometimes fussy
sometimes not
■ Doesn’t like anything even slightly
spicy, may cry
■ Loved the roast lamb.
■ Liked chickpeas once, hated them once
■ Is going to nanny’s house to sleep
straight after dinner. Has been over-
excited and on an edge since breakfast

Because [evidence?],
I believe
[making lamb and
chickpea stew] for
[us and the kids] will
result in [a great
dinner time and no
waste]. We’ll know
this is true if
[everyone eats and
ABORT!

@sonofswiss
What evidence
drove your idea?
How confident were
you in it?

@sonofswiss
4.Using hypotheses to
prioritise and persuade

Objective:
Simple to understand
Encourage non-
biased decision
making
Can be run as
collaborative
exercise
Text

@sonofswiss
Effort
Value
Confidence

@sonofswiss
Low value High value
High confidence
Low confidence

@sonofswiss
From evidence to confidence
■ Website data (AB, analytics)
■ Behavioural observation (eg in
usability testing)
■ Business data
■ Attitudinal observation (eg from
user interview)
■ Competitor analysis
■ Experience from other projects
■ Business ‘wants’
■ ‘Best practise’
■ Assumption/hunch
High
confidence
Low
confidence

@sonofswiss
Don’t
Maybe
Maybe
Definite
High confidence
Low confidence

@sonofswiss
Definite
No-brainer - proceed.
High confidence
Low confidence

@sonofswiss
High confidence
Low confidence
Don’t
Waste of time. Bin it.

@sonofswiss
Maybe
Consider proceeding
if low effort
High confidence
Low confidence

@sonofswiss
Maybe
OR Can the idea be
re-used in a different way
to make it high value?
High confidence
Low confidence

@sonofswiss
Maybe
OR gather more evidence
to increase confidence
High confidence
Low confidence

@sonofswiss
Never cook a lamb
and chickpea stew
for my kids.

Hypotheses: Supercharging your ideas and improving your decision making (NUX Manchester)

More Related Content

Similar to Hypotheses: Supercharging your ideas and improving your decision making (NUX Manchester)

Recently uploaded

Hypotheses: Supercharging your ideas and improving your decision making (NUX Manchester)

Editor's Notes