Discover how you can increase the performance and velocity of your testing program. Stephen will share his experience at Conversion.com, the UK’s biggest conversion optimisation agency. You’ll learn how to develop a process for personalisation (and avoid the mistakes most people make), how to accelerate your testing (with an advanced framework for prioritisation), and how to run advanced tests on product and price.

1. Habits of advanced conversion optimisers
Stephen Pavlovich
CEO @ Conversion.com

4. “To test it out, we tied together this fake
trailer where we combined 25 clips from all
these movies that inspired us.”
Matt Duffer

5. How most people see testing

9. Testing gives us a framework to
take risks and be more inventive.
@conversion_com

10. The changing nature of testing

11. “Almost any questions can be answered,
cheaply, quickly and finally, by a test campaign.
And that's the way to answer them –
not by arguments around a table.”

12. “Almost any questions can be answered,
cheaply, quickly and finally, by a test campaign.
And that's the way to answer them –
not by arguments around a table.”
Claude Hopkins
Scientific Advertising (1923)

13. Tests are important and
help us understand our
customers.
Good selling is based on
good testing.
Claude Hopkins
Scientific Advertising (1923)
“

20. WHAT WOULD YOU DO IF
LIFE HAD AN UNDO BUTTON?

21. Testing can be an undo button
for our most important decisions.
@conversion_com

22. Habits of testing
The hypothesis

23. 1. Find out why people
aren’t converting
2. Fix it.

24. 1. Find out why people
aren’t converting
2. Fix it.

25. Use this formula
to write your test
hypotheses instead...

26. We know that quant and qual data.
We believe that lever by test concept on area for
audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com

27. We know that quant and qual data.
We believe that lever by test concept on area for
audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com

28. We know that quant and qual data.
We believe that lever by test concept on area for
audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com

29. We know that quant and qual data.
asd

30. We know that quant and qual data.
asd
Sales data, web analytics, form
analysis, session replay, heatmapping,
onsite surveys, offsite surveys, focus
groups, usability tests...

31. We know that quant and qual data.
We know that 98% of visitors to the credit card
landing page don’t apply. In usability tests, all
users struggled to locate the call-to-action.
asd

33. We believe that lever by test concept...
asd

34. We believe that lever by test concept...
asd
Core “theme” of the test

35. We believe that lever by test concept...
asd
Core “theme” of the test
The implementation
of that lever

36. We believe that lever by test concept...
We believe that emphasising the call-to-action by
adding a button to the main content panel...
asd

37. ...on area for audience will result in goal.

38. ...on area for audience will result in goal.
The high-opportunity page
or element we’re testing

39. ...on area for audience will result in goal.
The high-opportunity page
or element we’re testing
The high-opportunity audience
segment we’re targeting

40. ...on area for audience will result in goal.
...on the credit card landing page for desktop
visitors will result in an increase in credit card
customers.

41. We’ll know by observing KPI for duration.

42. We’ll know by observing KPI for duration.
How we’ll measure
the goal

43. We’ll know by observing KPI for duration.
How we’ll measure
the goal
The expected test duration

44. We’ll know by observing KPI for duration.
We’ll know by observing successful 1-2-3 credit
card applications for 2 weeks.

45. Test hypothesis
We know that 98% of visitors to the credit card landing page don’t
apply. In usability tests, all users struggled to locate the
call-to-action.
We believe that emphasising the call-to-action by adding a button
to the main content panel on the credit card landing page for
desktop visitors will result in an increase in credit card customers.
We’ll know by observing successful 1-2-3 credit card applications
for 2 weeks.

46. We know that quant and qual data.
We believe that lever by test concept on area for
audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com

47. We know that quant and qual data.
We believe that lever by test concept on area for
audience will result in goal.
We’ll know by observing KPI for duration.
@conversion_com
Apply toanything

48. Habits of testing
Evolutionary testing

51. Lever
Lever 1
Lever 3
Lever 2

52. Increase test complexity
and apply to new areas
Iterate on
minimum
viable
test
Minimum
viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2

53. Increase test complexity
and apply to new areas
Iterate on
minimum
viable
test
Minimum
viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2

54. Increase test complexity
and apply to new areas
Iterate on
minimum
viable
test
Minimum
viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2

55. Increase test complexity
and apply to new areas
Iterate on
minimum
viable
test
Minimum
viable
test
Lever
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
Test
TestLever 1
Lever 3
Lever 2

56. Start with the
minimum viable test
then scale in reach and impact.
@conversion_com

57. Habits of testing
Every change is a test

59. 1810–1984

60. 1998–2016

61. 1995–2016

62. “If you double the number
of experiments you do per year
you’re going to
double your inventiveness.”
Jeff Bezos

63. Customer journey
TV
SEO
Display advertising
Social media
Outdoor advertising
PPC

64. Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Outdoor advertising
PPC

65. Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Outdoor advertising
PPC

66. Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Lifetime value
Referrals
Outdoor advertising
PPC

67. Customer journey
Website
App
In-store
TV
SEO
Call centreDisplay advertising
Social media
Product
Pricing
Lifetime value
Referrals
Outdoor advertising
PPC
Why arewe onlytestinghere?

68. Testing allows us to optimise
advertising, content, design,
functionality, price, product...
at scale.
@conversion_com

69. Before After

72. Sales
+100%

77. In a fast-changing world,
testing is our unfair advantage.
@conversion_com

78. @conversion_com
@pav
stephen.pavlovich@conversion.com
Thank you!