What does Issac Newton have to do with a cosmetics company discovering the key to online conversions? Join this session to find out. Top companies are now going beyond surface-level test results, and using experimentation to discover actionable insights about their customers that generate more than just clicks. In this session, Brooks Bell will share how to drive more impact from experimentation by uncovering the deeper insights behind the data. Attendees will learn: - Steps to create advanced customer theories that are iterative and transferable - A proven framework for deriving insights about customer behaviors - How to use those insights to own the customer narrative and exceed your KPIs

2. Defying Gravity: Channeling Newton
to Up-Level Your Experiments
Brooks Bell, CEO of Brooks Bell Inc.

3. Newton asked
WHY

4. We ask
What Works?

5. ?
?
?
?
??
? ?
?
?
?
?
? ?

6. Yet, we no longer
talk about WHY
in experimentation

7. Why are we afraid of WHY?
➔The famous Which Test Won game
➔Hundreds of unprovable possibilities
➔We really, really don’t like being wrong
➔If it works, who cares why?

8. “People shop for Projects, not Products”
Lowe’s looked for WHY

9. The opportunity is out there for us
➔We need to uncover the lasting
insight from our customers
➔Who owns the narrative wins

10. It starts with changing our
ALTITUDE

16. If it’s too high, it may not be useful
If it’s too low, it may not be transferable
When offering an answer to WHY...

17. Altitude is part of the
Brooks Bell Insight
Framework

18. Any idea of WHY it happened
=
customer theory

19. Customer theories
are categorized by
five altitudes
Basecamp
UI/Design/Platform
Jetstream
Personas
Satellite
Behavioral Econ Principles
Summit
State of Mind
Groundlevel
Outcome

20. Examples of Summit customer theories
“When it comes to
mortgages, people quickly
get overwhelmed with
information”
“Leisure travelers trust
their peers’ opinions
more than the hotel’s
marketing”
“Toy shoppers already
know which toy they
want to buy on mobile”

21. Altitude in action….
Control Variation 1
+2.3%
Numbers have been changed

22. Chat on Beauty site
Outcome:
Expanded Chat increases RPV by
2.3%
“Insight”:
Expanded Chat “works”
Recommendation:
Put Chat in more places
THE OLD WAY (Simplified)
Outcome:
Expanded Chat increases RPV by
2.3%
THE NEW WAY
WHY we think it works:
A few customer theories at
different altitudes

24. Altitude in action…
Control Variation 1
+2.3%

25. WHY does Chat work?
“Loss Aversion”: we are afraid of losing money
It is easier to see the box once it is expanded
Women are anxious about buying the wrong makeup and so they
need to be reassured by an expert to guide them
Women are looking to discover new product (rather than refilling old
product) and are less familiar with it

26. The right altitude makes it transferable
Women are anxious about buying the wrong makeup and so they
need to be reassured by an expert to guide them
Transferable customer theory
=
insight

27. (By the way….)

28. With 5+ experiments that
support it, an insight becomes a
KEY INSIGHT

29. Picture of family at a theme park

31. The current flow is low friction. Is that
helpful for a high consideration purchase?
Raises the questions:
• Does the price make this a higher consideration purchase?
• Do people want a low friction experience or is this overwhelming?
• Would creating commitment via higher friction help?

32. Whypothesis™:
If a 3-step purchase process wins, theme park visitors may be easily
easily overwhelmed with buying information due to pricey purchase
+20% Lift
in Revenue!

33. Because it won big, creating
customer theories & insights was
easy…

35. Test #2:
The current ticket navigation doesn’t show all
the ticket options.
Raises the questions:
• Would more people buy Annual Passes if it was more prominent?
• Is the toggle within the navigation intuitive or unhelpful?
• Would contrast and imagery make it more engaging?

36. Whypothesis™:
If a condensed navigation widget wins, theme park visitors may buy Annual
buy Annual Passes if we make it easier for them to discover them
2% Lift in CTR, but
3% Loss
in Revenue!

37. A losing test is the BEST time to ask
WHY!
- Unexpected results challenge confirmation bias
- Pain naturally drives learning
- A losing test can change your understanding of
your customer when you face it with humility

39. WHY did the condensed navigation lose?
New design may have made it harder to find the most popular
ticket option, due to contrast, lack of toggle and location
The visitor felt overwhelmed by the number of options available,
creating anxiety
Most visitors may not be interested buying Annual Passes online,
even if you make it easy to find
Only 3 more
supporting
experiments
for a Key
Insight!

40. A losing test is the BEST time to ask Why
- Unexpected results challenge your confirmation bias
- Pain naturally drives learning
- It helps you discover something new about your customer
(only if you address it head on!)

41. Key Insights are the opportunity
➔We need to uncover the lasting
insight from our customers
➔Who owns the narrative wins

43. Q&A

44. Interested in more details on the framework?
brooksbell.com/opticon18

45. Thank you!