YOU’RE TESTING
WHAT?!
FOUR TYPES OF TESTS
YOU SHOULD BE
RUNNING

MATT LACEY
Head of Optimisation

@matt_lacey12

17TH SEPT...
TALK OVERVIEW

What should you be testing?
Pricing & Price Framing
Proposition Testing

Optimising for Engagement
Optimisi...
WHAT SHOULD YOU BE TESTING?

@PRWD

#PRWDREVEAL
WHAT’S
WRONG WITH
TESTING
BUTTON
COLOURS

Disclosure: We test buttons. Size,
Prominence, Copy, Positioning and yes,
Colour...
HOW TO
CHOOSE WHAT
TO TEST

What is the data telling me about visitor
behaviour?
- Top landing pages
- Pages with high tra...
HOW TO
PRIORITISE
YOUR TESTS

Triage
-

Just Do It – JDI
Single Feature Test
Batch Testing
Radical Redesign Testing
Larger...
1. PRICE & PRICE FRAMING
TESTS

@PRWD

#PRWDREVEAL
WHY TEST
PRICING?

Through moderating or observing hundreds
of user testing sessions we have often
asked:

“If you could b...
SEVEN
PRINCIPLES OF
PRICING

1. Pricing should be based on value, not
cost
2. Prices should be tangible
3. Prices should b...
TECHNIQUE:

Three Principles of Anchoring

PRICE
ANCHORING

1. The perception of price is often based
on the context.
2. D...
WHICH TEST
WON?
Four Options:
1. Control
2. Variation A – (SAVE £22)
3. Variation B – (SAVE 38%)

4. Variation C – remove ...
PRICE & PRICE
FRAMING
TAKEAWAYS

Consider the range of price options
available and test.
• This could apply to product or ...
2. PROPOSITION TESTING

@PRWD

#PRWDREVEAL
The second a visitor comes to you they have
in mind: “Why should I buy from you versus
all your competitors?”
You need to ...
BEFORE

AFTER

4.35% increase in CR
This equated to a six-figure increase in revenue.

@PRWD

#PRWDREVEAL
GET REVISING
SIGN-UP TABLE
Price doesn’t put me off, although the
basic package is more than enough for
me.
I think the si...
@PRWD

#PRWDREVEAL
GET REVISING
SIGN-UP TABLE

185%

increase in premium subscriptions

with a small drop in free subscription. This was seen...
PROPOSITION
TESTING
TAKEAWAYS

Test what features/functionality is given
for free and what is paid for
Test how your propo...
3 RADICAL REDESIGNS
OPTIMISING FOR ENGAGEMENT

@PRWD

#PRWDREVEAL
OPTIMISING FOR
ENGAGEMENT

This opens up the top of the conversion
funnel.
This is likely to be the point where you
have t...
BEFORE

AFTER

10.2%

performance

improvement in booking conversion rate

@PRWD

#PRWDREVEAL
BEFORE

AFTER

10% improvement in CR
@PRWD

#PRWDREVEAL
BEFORE

AFTER

6.8% improvement in CR
@PRWD

#PRWDREVEAL
ENGAGEMENT
OPTIMISATION
TAKEAWAYS

Let users know that they have landed in the
right place
Make sure that pages have a cle...
4 RADICAL REDESIGNS
OPTIMISING FOR CONVERSION

@PRWD

#PRWDREVEAL
OPTIMISING FOR
CONVERSIONS

Testing towards the end of your funnel
helps to convert users who are closest to
converting an...
BEFORE

AFTER

72% improvement in
Registrations from the homepage

@PRWD

#PRWDREVEAL
BEFORE

AFTER

7.4% CR improvement from
sign-up page

@PRWD

#PRWDREVEAL
CONVERSION
OPTIMISATION
TAKEAWAYS

Don’t be afraid to offer visitors an
opportunity to convert at an early stage
Use persu...
FINAL
TAKEAWAY

With the right kind of research
and preparation the odds are
stacked in your favour.
Therefore:

Be data d...
THANKS FOR
LISTENING
Are there any questions?

MATT LACEY
17TH SEPTEMBER 2013

Head of Optimisation

@matt_lacey12

@PRWD
...
Upcoming SlideShare
Loading in …5
×

You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

984 views

Published on

Matt Lacey Head of Optimisation at PRWD

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
984
On SlideShare
0
From Embeds
0
Number of Embeds
130
Actions
Shares
0
Downloads
7
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

You're testing what?! Four Types of Tests You Should be Running. Matt Lacey, Head of Optimisation at PRWD

  1. 1. YOU’RE TESTING WHAT?! FOUR TYPES OF TESTS YOU SHOULD BE RUNNING MATT LACEY Head of Optimisation @matt_lacey12 17TH SEPTEMBER 2013 PRWD Reveal MANCHESTER
  2. 2. TALK OVERVIEW What should you be testing? Pricing & Price Framing Proposition Testing Optimising for Engagement Optimising for Conversion @PRWD #PRWDREVEAL
  3. 3. WHAT SHOULD YOU BE TESTING? @PRWD #PRWDREVEAL
  4. 4. WHAT’S WRONG WITH TESTING BUTTON COLOURS Disclosure: We test buttons. Size, Prominence, Copy, Positioning and yes, Colour. But: There are likely to be constraints that limit the number of tests that you can run based on design & developer resource In most situations we find that there are tests that can be run that will deliver more interesting results and more value @PRWD #PRWDREVEAL
  5. 5. HOW TO CHOOSE WHAT TO TEST What is the data telling me about visitor behaviour? - Top landing pages - Pages with high traffic - Key leak or conversion points What are users telling me about their experiences? What can we observe about their behaviour? - Usability errors - Influential proposition messaging What does the business know about customer struggles or business challenges? - Customer service insights - Merchandising team @PRWD #PRWDREVEAL
  6. 6. HOW TO PRIORITISE YOUR TESTS Triage - Just Do It – JDI Single Feature Test Batch Testing Radical Redesign Testing Larger Redesign projects Triangulation - Multiple research methods confirm or strengthen the same issues Potential Impact vs Effort Required - Likelihood to impact key metrics vs - Design/Development effort - Build time - Development schedule @PRWD #PRWDREVEAL
  7. 7. 1. PRICE & PRICE FRAMING TESTS @PRWD #PRWDREVEAL
  8. 8. WHY TEST PRICING? Through moderating or observing hundreds of user testing sessions we have often asked: “If you could buy the same product from two different websites, what factors would be important in helping you choose which to buy from?” User consistently respond with: 1. Price 2. Delivery ( price and timescales) @PRWD #PRWDREVEAL
  9. 9. SEVEN PRINCIPLES OF PRICING 1. Pricing should be based on value, not cost 2. Prices should be tangible 3. Prices should be comparable 4. To change your prices, you must reframe 5. Price differentiation is the key enabler of profit 6. Pricing communication shapes the customers perception of value 7. You must be prepared to lose some sales. S o u rce: L ei g h Cal dwel l : T h e P s y ch o l o g y o f P ri ce: H o w t o u s e p ri ce t o i n creas e deman d, p ro f i t an d cu s t o mer s at i s f act i o n @PRWD #PRWDREVEAL
  10. 10. TECHNIQUE: Three Principles of Anchoring PRICE ANCHORING 1. The perception of price is often based on the context. 2. Decision making and the power of suggestion 3. Avoiding extremes S o u rce: P ri ce In t el l i g en t l y http://blog.priceintelligently.com/blog/bid/181199/PriceAnchoring-to-Optimize-Your-Pricing-Strategy @PRWD #PRWDREVEAL
  11. 11. WHICH TEST WON? Four Options: 1. Control 2. Variation A – (SAVE £22) 3. Variation B – (SAVE 38%) 4. Variation C – remove ‘(cancel anytime)’ from monthly 12.5% overall increase in subscriptions for Variation A (SAVE £22) @PRWD #PRWDREVEAL
  12. 12. PRICE & PRICE FRAMING TAKEAWAYS Consider the range of price options available and test. • This could apply to product or service costs, delivery costs, etc Run tests to make price more transparent • What do people get for their money • Test messaging that explains what people are paying for. Control comparisons • Test using competitor prices for comparisons @PRWD #PRWDREVEAL
  13. 13. 2. PROPOSITION TESTING @PRWD #PRWDREVEAL
  14. 14. The second a visitor comes to you they have in mind: “Why should I buy from you versus all your competitors?” You need to display clear unique value propositions or unique campaign propositions. Bryan Eisenberg, Godfather of CRO Everyone who is successful in online conversion is doing this. If you’re not, you’re wasting your time. Watch clip: http://www.youtube.com/watch?feature =player_embedded&v =ps7bB9-ksMY @PRWD #PRWDREVEAL
  15. 15. BEFORE AFTER 4.35% increase in CR This equated to a six-figure increase in revenue. @PRWD #PRWDREVEAL
  16. 16. GET REVISING SIGN-UP TABLE Price doesn’t put me off, although the basic package is more than enough for me. I think the site has already got everything I need. Moderated User testing participants @PRWD #PRWDREVEAL
  17. 17. @PRWD #PRWDREVEAL
  18. 18. GET REVISING SIGN-UP TABLE 185% increase in premium subscriptions with a small drop in free subscription. This was seen to be a good trade-off. You have more features held back to encourage people to subscribe at a later date. @PRWD #PRWDREVEAL
  19. 19. PROPOSITION TESTING TAKEAWAYS Test what features/functionality is given for free and what is paid for Test how your proposition is displayed/framed on your site Test offering different thresholds for free delivery Test introducing urgency to the process @PRWD #PRWDREVEAL
  20. 20. 3 RADICAL REDESIGNS OPTIMISING FOR ENGAGEMENT @PRWD #PRWDREVEAL
  21. 21. OPTIMISING FOR ENGAGEMENT This opens up the top of the conversion funnel. This is likely to be the point where you have the largest number of potential test participants. Search for landing or key site pages with high bounce rates or those that rarely lead to conversions Key Areas to Test Homepage Top landing pages @PRWD #PRWDREVEAL
  22. 22. BEFORE AFTER 10.2% performance improvement in booking conversion rate @PRWD #PRWDREVEAL
  23. 23. BEFORE AFTER 10% improvement in CR @PRWD #PRWDREVEAL
  24. 24. BEFORE AFTER 6.8% improvement in CR @PRWD #PRWDREVEAL
  25. 25. ENGAGEMENT OPTIMISATION TAKEAWAYS Let users know that they have landed in the right place Make sure that pages have a clear focus and visual hierarchy. Users need to know where to begin. Test both simplifying the experience and adding in more detail A smaller amount of highly qualified visitors can trump large amounts of unqualified visitors and convert better @PRWD #PRWDREVEAL
  26. 26. 4 RADICAL REDESIGNS OPTIMISING FOR CONVERSION @PRWD #PRWDREVEAL
  27. 27. OPTIMISING FOR CONVERSIONS Testing towards the end of your funnel helps to convert users who are closest to converting and tend to have higher intent. Improvements here opens up the bottom of the conversion funnel. Simplifying and removing issues that slow momentum is crucial. Key Areas to Test Checkout (login, delivery, payment) Contact/Sign-up forms @PRWD #PRWDREVEAL
  28. 28. BEFORE AFTER 72% improvement in Registrations from the homepage @PRWD #PRWDREVEAL
  29. 29. BEFORE AFTER 7.4% CR improvement from sign-up page @PRWD #PRWDREVEAL
  30. 30. CONVERSION OPTIMISATION TAKEAWAYS Don’t be afraid to offer visitors an opportunity to convert at an early stage Use persuasive techniques to pull users through (not push!) Simplify forms to the bare minimum fields required Reduce errors. There’s a huge correlation between users getting an error and abandoning the funnel Provide key supporting proposition messaging throughout the process Re-enforce positive messages (e.g. security in checkout) @PRWD #PRWDREVEAL
  31. 31. FINAL TAKEAWAY With the right kind of research and preparation the odds are stacked in your favour. Therefore: Be data driven. Be bold. High risk, high reward. @PRWD #PRWDREVEAL
  32. 32. THANKS FOR LISTENING Are there any questions? MATT LACEY 17TH SEPTEMBER 2013 Head of Optimisation @matt_lacey12 @PRWD #PRWDREVEAL PRWD Reveal MANCHESTER

×