Slides from my Mind The Product London 2018 talk about learning, "failing", and framing your thinking as hypotheses (for running experiments, A/B tests, or just exploring a product space). More details about #experimentation and #abtesting here: http://experimentationhub.com/a-b-sensei.html

1. Get comfortable breaking your product*
*in the right way!
Rik Higham
@rikhigham
ExperimentationHub.com

4. It only ‘failed’ if you fail to learn

5. “every failure told me something that I was able to
incorporate into the next attempt.” – Thomas Edison

7. “Every child is an artist. The problem is how to
remain an artist once we grow up.” – Pablo Picasso

11. Here’s the Drop™

13. Hypothesis driven development?
Break mindfully?
Explore how to get to vision ?
Frame our thinking
Break things mindfully
Frame your thinking as a hypothesis

14. Scientist, Psychologist,
Artist, & master of none.
You:

16. Causality
Insights

17. Per capita cheese consumption
correlates with
Number of people who died by becoming tangled in their bedsheets
Bedsheet tanglings Cheese consumed
tylervigen.com

18. Insights
Causality Testable
Falsifiable

21. Based on [quantitative/qualitative insight].
We predict that [product change] will cause [impact].

22. Based on data that shows 70% of people fill out 2 or
3 fields before bouncing.
We predict that reducing the number of input fields
on the registration form to just ‘email’ and ‘password’
will cause more people to sign up.

25. Hope we'll soon get
another website with
the facilities the
OLD
Skycanner had!

27. I could always see how much flights cost from all over the month.
Now I see a nightmare.
Now I have to do dozens of searches to see the price of the month.
A nightmare,
a nightmare,
and once again a nightmare.
Who came up with this nightmare?

28. “Experiments don’t fail; hypotheses are proven wrong.”
– Airbnb

30. Think like a scientist

32. Think like a scientist
Try and prove yourself wrong

35. wtf did you do?! I used to like clicking
through the cities, it was so easy.
It was my favourite thing to do
on the crapper!

36. Don’t Listen to Your Users, by Gillian Morris: https://bit.ly/1UsAsqq
Retention Rate by Month

37. “Web geek”

38. Hypothesis Kit
The Hypothesis Kit was developed by Rik Higham and Colin McFarland, with contributions from David Pier, Lukas Vermeer, Ya Xu, Ronny Kohavi, Craig Sullivan, and Jane Murison.
Based on [quantitative/qualitative insight].
We predict that [product change] will cause [impact].
We will test this by assuming the change has no effect (the null
hypothesis) and running an experiment for [X business cycles].
If we can measure a [Y%] statistically significant change in [metric]
then we reject the null hypothesis and conclude there is an effect.
Design like you’re right:
Test like you’re wrong:

39. ExperimentationHub.com
Skip Ad

40. Design like you’re right
Test like you’re wrong
ExperimentationHub.com @rikhigham
Think like a scientist
Focus on your insights
Stop failing, and start learning