@paulrouke #elitecamp2015
FROM ITERATIVE TO INNOVATIVE TESTING
EXPLOITING THE FULL SPECTRUM
OF TESTING OPPORTUNITIES
PAUL ...
@paulrouke #elitecamp2015
ITERATIVE
TESTING
INNOVATIVE
TESTING
OPPORTUNITY
@paulrouke #elitecamp2015
OUT-GROW YOUR
COMPETITION
By embracing the full spectrum of
research & insight driven
testing op...
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
A full service conversion optimisation agency based in the UK
Optimising online experiences sinc...
@paulrouke #elitecamp2015
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
WHAT IS
THE DIFFERENCE?
@paulrouke #elitecamp2015
ITERATIVE TESTING
Headline changes
Copy changes
Image changes
Button size changes
Button text ch...
@paulrouke #elitecamp2015
Control
ITERATIVE TESTING
@paulrouke #elitecamp2015
PRWD Test Variation
ITERATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 4.7%Reply Conversion Rate
ITERATIVE TESTING
@paulrouke #elitecamp2015
Control
ITERATIVE TESTING
@paulrouke #elitecamp2015
PRWD Variation
ITERATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 20%Brochure Requests
CONSISTENTLY COMMUNICATE HOW GOOD YOU ARE
ITERATIVE TESTING
@paulrouke #elitecamp2015
Ad-hoc testing with weak hypotheses
will destroy buy-in
Intelligent, insight driven iterative te...
@paulrouke #elitecamp2015
INNOVATIVE TESTING
Full redesigns of homepage
Full redesigns of primary template pages
Landing p...
@paulrouke #elitecamp2015
Control
INNOVATIVE TESTING
@paulrouke #elitecamp2015
PRWD Test Variation
INNOVATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 78%New visitor sign-ups
BE BOLD & PROVIDE EARLY & EASY OPPORTUNITIES TO CONVERT
INNOVAT...
@paulrouke #elitecamp2015
Poorly planned innovative testing will waste
valuable resources & damage testing momentum
Intell...
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
WHAT ARE THE
INFLUENCING FACTORS?
@paulrouke #elitecamp2015
ITERATIVE TESTING
@paulrouke #elitecamp2015
INNOVATIVE TESTING
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
WHY CHOOSE
EACH TYPE?
@paulrouke #elitecamp2015
Quick wins
Start to learn
Use your tool
Limited resources
Political business
Build momentum
Demo...
@paulrouke #elitecamp2015
Control
ITERATIVE TESTING
@paulrouke #elitecamp2015
PRWD Test Variation
ITERATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 4.3%Sitewide Purchase Conversion Rate
Over £250k in additional revenue
ITERATIVE TESTING
@paulrouke #elitecamp2015
Make significant increases
Really shift user behaviour
Quantifiable evidence for a major change
...
@paulrouke #elitecamp2015
BUSINESS CHALLENGE
Identify the 1 most influential feature why
visitors will choose Premium over...
@paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?
@paulrouke #elitecamp2015
GET REVISING
FREE OR PREMIUM DECISION
Price doesn’t put me off, although the
basic package is mo...
@paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?
@paulrouke #elitecamp2015
GET REVISING
SIGN-UP TABLE
+ 185%Premium Subscriptions
- 10%
Free Sign-ups
WHY CHOOSE INNOVATIVE...
@paulrouke #elitecamp2015
WHY CHOOSE INNOVATIVE TESTING?
@paulrouke #elitecamp2015
Intelligent, innovative testing is essential
in exploiting your growth potential
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
WHEN ARE YOU BEST
CHOOSING EACH TYPE?
@paulrouke #elitecamp2015
Quick win test opportunities
Primary proposition headlines
Primary CTA’s in to sign-up process
W...
@paulrouke #elitecamp2015
When you have bold and/or innovative ideas built off previous test insights & deep user understa...
@paulrouke #elitecamp2015
Control
INNOVATIVE TESTING
@paulrouke #elitecamp2015
PRWD Test
Variation
INNOVATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 18%Brochure Requests
INNOVATIVE TESTING
@paulrouke #elitecamp2015
Control
INNOVATIVE TESTING
@paulrouke #elitecamp2015
PRWD Test
Variation
INNOVATIVE TESTING
@paulrouke #elitecamp2015
Result
+ 31%Brochure Requests
INNOVATIVE TESTING
@paulrouke #elitecamp2015
What are the 2 biggest influencers to
running successful innovative tests?
1) In-depth user unde...
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
WHAT BUSINESS IMPACT
DOES EACH TYPE HAVE?
@paulrouke #elitecamp2015
BUSINESS IMPACT THROUGH ITERATIVE TESTING
@paulrouke #elitecamp2015
BUSINESS IMPACT THROUGH INNOVATIVE TESTING
@paulrouke #elitecamp2015
Innovative Testing
Iterative Testing
User Research
High Quality & Intelligent
Optimisation Strat...
@paulrouke #elitecamp2015
Innovative Testing
Iterative Testing
User Research
=High Quality & Intelligent
Optimisation Stra...
@paulrouke #elitecamp2015
What is the difference?
What are the influencing factors?
Why choose each type?
When are you bes...
@paulrouke #elitecamp2015
8 STEPS TO OUT-GROW
YOUR COMPETITION
@paulrouke #elitecamp2015
Foundations - get the right foundations in place for optimisation
Quality – focus on quality of ...
PAUL ROUKE
Founder & Director of
Optimisation
@paulrouke
THANK YOU FOR LISTENING
HERE’S MY FINAL CTA…
OVER 75 CRO ARTICLES...
Upcoming SlideShare
Loading in …5
×

Iterative versus innovative testing: Exploiting the Full Spectrum of Testing Opportunities - Paul rouke at Elite Camp 2015

2,718 views

Published on

These are the slides from Paul Rouke's talk 'From Iterative to Innovative Testing' which he gave at Elite Camp 2015 in Talinn, Estonia.

How progressive are your tests? Is radical testing part of your company culture? How do you use testing as part of a website redesign? Do you test business level hypotheses?

Gain a first hand understanding of when to do innovative testing, how you can develop innovative test hypotheses, and see how companies are changing their culture and whole approach to testing for growth.

Paul Rouke is the Founder & Director of Optimisation at PRWD.

Elite Camp is a highly practical event. No fluff, no theoretical debates, just an event which provides 100% practical, actionable content, delivered by highly-sought after experts, and infused with hardcore networking.

Published in: Retail
2 Comments
8 Likes
Statistics
Notes
  • Hi Paul, this is very interesting. However, if your 'innovative' AB test shows negative results, how can you find what went wrong as you have changed so much. All your examples show positive results but it is probably not always the case?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thanks, Paul - an education! Professor Hugh Wilson, Cranfield School of Management
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,718
On SlideShare
0
From Embeds
0
Number of Embeds
91
Actions
Shares
0
Downloads
30
Comments
2
Likes
8
Embeds 0
No embeds

No notes for slide

Iterative versus innovative testing: Exploiting the Full Spectrum of Testing Opportunities - Paul rouke at Elite Camp 2015

  1. 1. @paulrouke #elitecamp2015 FROM ITERATIVE TO INNOVATIVE TESTING EXPLOITING THE FULL SPECTRUM OF TESTING OPPORTUNITIES PAUL ROUKE Founder & Director of Optimisation at PRWD
  2. 2. @paulrouke #elitecamp2015 ITERATIVE TESTING INNOVATIVE TESTING OPPORTUNITY
  3. 3. @paulrouke #elitecamp2015 OUT-GROW YOUR COMPETITION By embracing the full spectrum of research & insight driven testing opportunities
  4. 4. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  5. 5. @paulrouke #elitecamp2015 A full service conversion optimisation agency based in the UK Optimising online experiences since 2004 Represents the UK in the Global Conversion Alliance, alongside WiderFunnel, WebArts & Conversionista Conducted over 600 hours of in-depth user research across multiple sectors 540% ROI delivered on average through PRWD’s Six Step Optimisation Method ABOUT PRWD
  6. 6. @paulrouke #elitecamp2015
  7. 7. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  8. 8. @paulrouke #elitecamp2015 WHAT IS THE DIFFERENCE?
  9. 9. @paulrouke #elitecamp2015 ITERATIVE TESTING Headline changes Copy changes Image changes Button size changes Button text changes Content removal Page layout changes Removing or optimising form fields Go on then if you must… button colour changes IN SUMMARY Almost all the testing that currently takes place
  10. 10. @paulrouke #elitecamp2015 Control ITERATIVE TESTING
  11. 11. @paulrouke #elitecamp2015 PRWD Test Variation ITERATIVE TESTING
  12. 12. @paulrouke #elitecamp2015 Result + 4.7%Reply Conversion Rate ITERATIVE TESTING
  13. 13. @paulrouke #elitecamp2015 Control ITERATIVE TESTING
  14. 14. @paulrouke #elitecamp2015 PRWD Variation ITERATIVE TESTING
  15. 15. @paulrouke #elitecamp2015 Result + 20%Brochure Requests CONSISTENTLY COMMUNICATE HOW GOOD YOU ARE ITERATIVE TESTING
  16. 16. @paulrouke #elitecamp2015 Ad-hoc testing with weak hypotheses will destroy buy-in Intelligent, insight driven iterative testing will build your test & learn culture
  17. 17. @paulrouke #elitecamp2015 INNOVATIVE TESTING Full redesigns of homepage Full redesigns of primary template pages Landing page redesigns Customer journey changes Testing changes to a business proposition Tests prior to changing back-end systems Reducing steps within a flow ie. Checkout, application, registration Even full website redesigns IN SUMMARY Very few businesses are bold enough to invest in this type of testing
  18. 18. @paulrouke #elitecamp2015 Control INNOVATIVE TESTING
  19. 19. @paulrouke #elitecamp2015 PRWD Test Variation INNOVATIVE TESTING
  20. 20. @paulrouke #elitecamp2015 Result + 78%New visitor sign-ups BE BOLD & PROVIDE EARLY & EASY OPPORTUNITIES TO CONVERT INNOVATIVE TESTING
  21. 21. @paulrouke #elitecamp2015 Poorly planned innovative testing will waste valuable resources & damage testing momentum Intelligent, insight driven innovative testing can help transform & grow your business
  22. 22. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  23. 23. @paulrouke #elitecamp2015 WHAT ARE THE INFLUENCING FACTORS?
  24. 24. @paulrouke #elitecamp2015 ITERATIVE TESTING
  25. 25. @paulrouke #elitecamp2015 INNOVATIVE TESTING
  26. 26. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  27. 27. @paulrouke #elitecamp2015 WHY CHOOSE EACH TYPE?
  28. 28. @paulrouke #elitecamp2015 Quick wins Start to learn Use your tool Limited resources Political business Build momentum Demonstrate ROI Develop a culture IN SUMMARY You want to walk before you start running with testing WHY CHOOSE ITERATIVE TESTING?
  29. 29. @paulrouke #elitecamp2015 Control ITERATIVE TESTING
  30. 30. @paulrouke #elitecamp2015 PRWD Test Variation ITERATIVE TESTING
  31. 31. @paulrouke #elitecamp2015 Result + 4.3%Sitewide Purchase Conversion Rate Over £250k in additional revenue ITERATIVE TESTING
  32. 32. @paulrouke #elitecamp2015 Make significant increases Really shift user behaviour Quantifiable evidence for a major change Prior to back-end system changes Business proposition evaluation Reached local maximum Value proposition learning Simplify a stepped process Data driven website redesign WHY CHOOSE INNOVATIVE TESTING? IN SUMMARY You want grow your business rather than just optimise your website performance
  33. 33. @paulrouke #elitecamp2015 BUSINESS CHALLENGE Identify the 1 most influential feature why visitors will choose Premium over FREE
  34. 34. @paulrouke #elitecamp2015 WHY CHOOSE INNOVATIVE TESTING?
  35. 35. @paulrouke #elitecamp2015 GET REVISING FREE OR PREMIUM DECISION Price doesn’t put me off, although the basic package is more than enough for me. I think the site has already got everything I need. Moderated user research participant WHY CHOOSE INNOVATIVE TESTING?
  36. 36. @paulrouke #elitecamp2015 WHY CHOOSE INNOVATIVE TESTING?
  37. 37. @paulrouke #elitecamp2015 GET REVISING SIGN-UP TABLE + 185%Premium Subscriptions - 10% Free Sign-ups WHY CHOOSE INNOVATIVE TESTING?
  38. 38. @paulrouke #elitecamp2015 WHY CHOOSE INNOVATIVE TESTING?
  39. 39. @paulrouke #elitecamp2015 Intelligent, innovative testing is essential in exploiting your growth potential
  40. 40. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  41. 41. @paulrouke #elitecamp2015 WHEN ARE YOU BEST CHOOSING EACH TYPE?
  42. 42. @paulrouke #elitecamp2015 Quick win test opportunities Primary proposition headlines Primary CTA’s in to sign-up process When imagery is key Sitewide value proposition messages Landing pages with forms Form tests WHEN TO CHOOSE ITERATIVE TESTING IN SUMMARY When you have non-optimised primary pages in your user journey
  43. 43. @paulrouke #elitecamp2015 When you have bold and/or innovative ideas built off previous test insights & deep user understanding If you have run a range of iterative tests on a key page and are now struggling to impact page performance After you have run tests aimed at understanding what USP’s most resonate with your target audience When there is an opportunity to deliver a radically different user experience aimed at simplifying a process ie. Registration, checkout, application When there will be a considerable development cost to fully implementing a proposed solution When you have quite low traffic and want to reduce your test length When data driven optimisation is truly part of your company culture WHEN TO CHOOSE INNOVATIVE TESTING When you are ready to grow your business and not just optimise your website performance
  44. 44. @paulrouke #elitecamp2015 Control INNOVATIVE TESTING
  45. 45. @paulrouke #elitecamp2015 PRWD Test Variation INNOVATIVE TESTING
  46. 46. @paulrouke #elitecamp2015 Result + 18%Brochure Requests INNOVATIVE TESTING
  47. 47. @paulrouke #elitecamp2015 Control INNOVATIVE TESTING
  48. 48. @paulrouke #elitecamp2015 PRWD Test Variation INNOVATIVE TESTING
  49. 49. @paulrouke #elitecamp2015 Result + 31%Brochure Requests INNOVATIVE TESTING
  50. 50. @paulrouke #elitecamp2015 What are the 2 biggest influencers to running successful innovative tests? 1) In-depth user understanding 2) Meaningful insights from prior tests
  51. 51. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  52. 52. @paulrouke #elitecamp2015 WHAT BUSINESS IMPACT DOES EACH TYPE HAVE?
  53. 53. @paulrouke #elitecamp2015 BUSINESS IMPACT THROUGH ITERATIVE TESTING
  54. 54. @paulrouke #elitecamp2015 BUSINESS IMPACT THROUGH INNOVATIVE TESTING
  55. 55. @paulrouke #elitecamp2015 Innovative Testing Iterative Testing User Research High Quality & Intelligent Optimisation Strategy =
  56. 56. @paulrouke #elitecamp2015 Innovative Testing Iterative Testing User Research =High Quality & Intelligent Optimisation Strategy
  57. 57. @paulrouke #elitecamp2015 What is the difference? What are the influencing factors? Why choose each type? When are you best choosing each type? What business impact does each type deliver? 8 steps to out-grow your competition FROM ITERATIVE TO INNOVATIVE TESTING
  58. 58. @paulrouke #elitecamp2015 8 STEPS TO OUT-GROW YOUR COMPETITION
  59. 59. @paulrouke #elitecamp2015 Foundations - get the right foundations in place for optimisation Quality – focus on quality of tests, then quantity Intelligence - start with intelligent, research driven simple tests Identification – use simple test insights & research to identify radical test opportunities Momentum - build up testing momentum, buy-in and awareness of learnings Business case – use ROI from simple/iterative tests to provide the business case for radical testing Go innovative - begin planning and delivering bold, innovative tests alongside iterative tests Growth - grow faster than your competition by exploiting the full spectrum of testing opportunities 1 2 3 4 5 6 7 8 STEPS TO EXPLOIT THE FULL SPECTRUM OF TESTING OPPORTUNITIES 8
  60. 60. PAUL ROUKE Founder & Director of Optimisation @paulrouke THANK YOU FOR LISTENING HERE’S MY FINAL CTA… OVER 75 CRO ARTICLES bit.ly/CROresources

×