YOU’RE TESTING
WHAT?!
FOUR TYPES OF TESTS
YOU SHOULD BE
RUNNING
MATT LACEY
Head of Optimisation
@matt_lacey12
27TH MARCH 2...
@matt_lacey12
 #CampDigital
Head of Optimisation at PRWD
One of the UK’s most respected CRO agencies
Lead Optimisation Pro...
@matt_lacey12
 #CampDigital
@matt_lacey12
 #CampDigital
What should we be testing?
Pricing & Price Framing
Proposition Testing
Optimising for Engageme...
@matt_lacey12
 #CampDigital
WHAT SHOULD YOU BE TESTING?
@matt_lacey12
 #CampDigital
WHAT’S
WRONG WITH
TESTING
BUTTON
COLOURS
Disclosure: We test buttons. Size,
Prominence, Copy, ...
@matt_lacey12
 #CampDigital
SECTION TITLE
THE CRO PROCESS
@matt_lacey12
 #CampDigital
SECTION TITLE
WHAT DOES IT
INVOLVE?
Analytics
User Research
Cross Device User Testing
Hypothes...
@matt_lacey12
 #CampDigital
What is the data telling me about visitor
behaviour?
• Top landing pages
• Pages with high tra...
@matt_lacey12
 #CampDigital
Triage
• Just Do It – JDI
• Single Feature Test
• Batch Testing
• Radical Redesign Testing
• L...
@matt_lacey12
 #CampDigital
1. PRICE & PRICE FRAMING TESTS
@matt_lacey12
 #CampDigital
Through moderating or observing hundreds
of user testing sessions we have often
asked:
WHY TES...
@matt_lacey12
 #CampDigital
1.  Pricing should be based on value, not
cost
2.  Prices should be tangible
3.  Prices should...
@matt_lacey12
 #CampDigital
Three Principles of Anchoring
1.  The perception of price is often based
on the context.
2.  D...
@matt_lacey12
 #CampDigital
Four Options:
1. Control
2. Variation A – (SAVE £22)
3. Variation B – (SAVE 38%)
4. Variation ...
@matt_lacey12
 #CampDigital
1.  Control - Annual £39.99
(Save £44)
2. Variation – £3.33 per
month (£39.99 annually)
WHICH ...
@matt_lacey12
 #CampDigital
Consider the range of price options
available and test
• This could apply to product or servic...
@matt_lacey12
 #CampDigital
2. PROPOSITION TESTING
@matt_lacey12
 #CampDigital
Watch clip: http://www.youtube.com/watch?feature=player_embedded&v=ps7bB9-ksMY
“
The second a ...
@matt_lacey12
 #CampDigital
We can define this as 'Why should I buy
from you'. Typically this is a short
statement that is ...
@matt_lacey12
 #CampDigital
GLOBAL HEADER UVP
@matt_lacey12
 #CampDigital
USPs - Unique Selling Proposition
USPs are more focused on the shopping
experience and what ad...
@matt_lacey12
 #CampDigital
PRODUCT PAGE USP
@matt_lacey12
 #CampDigital
BASKET PAGE USPS
@matt_lacey12
 #CampDigital
Unique Campaign Propositions - 
UCP’s are messages based on specific
campaigns or promotions. T...
@matt_lacey12
 #CampDigital
UNIQUE CAMPAIGN PROPOSITIONS
@matt_lacey12
 #CampDigital
THE NORTH FACE
USP BAR TEST
@matt_lacey12
 #CampDigital
BEFORE
AFTER
4.35% increase in CR
This equated to a six-figure increase in revenue.
@matt_lacey12
 #CampDigital
GET REVISING
SIGN-UP TABLE TEST
@matt_lacey12
 #CampDigital
GET REVISING
SIGN-UP TABLE
“
Price doesn’t put me off, although the
basic package is more than...
@matt_lacey12
 #CampDigital
@matt_lacey12
 #CampDigital
GET REVISING
SIGN-UP TABLE
185%increase in premium subscriptions
with a small drop in free sub...
@matt_lacey12
 #CampDigital
Test what features/functionality is given
for free and what is paid for
Test how your proposit...
@matt_lacey12
 #CampDigital
3. RADICAL REDESIGNS
OPTIMISING FOR ENGAGEMENT
@matt_lacey12
 #CampDigital
This opens up the top of the conversion
funnel. 
This is likely to be the point where you have...
@matt_lacey12
 #CampDigital
RUSH HAIR
HOMEPAGE TEST
@matt_lacey12
 #CampDigital
BEFORE AFTER
10.2%performance
improvement in booking conversion rate
@matt_lacey12
 #CampDigital
MARKED BY TEACHERS
ESSAY DETAIL PAGE
@matt_lacey12
 #CampDigital
BEFORE AFTER
10% improvement in CR
@matt_lacey12
 #CampDigital
BEFORE AFTER
6.8% improvement in CR
@matt_lacey12
 #CampDigital
Let users know that they have landed in the
right place
Make sure that pages have a clear focu...
@matt_lacey12
 #CampDigital
4. RADICAL REDESIGNS
OPTIMISING FOR CONVERSION
@matt_lacey12
 #CampDigital
Testing towards the end of your funnel
helps to convert users who are closest to
converting an...
@matt_lacey12
 #CampDigital
THE STUDENT ROOM
HOMEPAGE TEST
@matt_lacey12
 #CampDigital
BEFORE AFTER
72% improvement in
Registrations from the homepage
@matt_lacey12
 #CampDigital
BEFORE AFTER
7.4% CR improvement from
sign-up page
@matt_lacey12
 #CampDigital
Don’t be afraid to offer visitors an
opportunity to convert at an early stage
Use persuasive t...
@matt_lacey12
 #CampDigital
With the right kind of research
and preparation the odds are
stacked in your favour. 
Therefor...
@matt_lacey12
 #CampDigital
Conversion optimisation case studies
Usability best practice
User research insights
CRO presen...
@matt_lacey12
 #CampDigital
Are there any questions?
THANKS FOR
LISTENING
MATT LACEY
Head of Optimisation
@matt_lacey12
Upcoming SlideShare
Loading in …5
×

You're testing what!

7,017 views

Published on

Matt presents the four types of test every Ecommerce business should be running. He explains how to identify what to test and how to prioritise your testing schedule, as well as settling once and for all the question “what’s wrong with testing button colours”? He’ll cover pricing and price framing, proposition testing and radical redesigns to optimise for engagement and conversion. Key take-aways for each section are shared ready for businesses to go and implement in their own organisation.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
7,017
On SlideShare
0
From Embeds
0
Number of Embeds
56
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

You're testing what!

  1. 1. YOU’RE TESTING WHAT?! FOUR TYPES OF TESTS YOU SHOULD BE RUNNING MATT LACEY Head of Optimisation @matt_lacey12 27TH MARCH 2014 CAMP DIGITAL MANCHESTER
  2. 2. @matt_lacey12 #CampDigital Head of Optimisation at PRWD One of the UK’s most respected CRO agencies Lead Optimisation Programmes for The North Face, Student Room & Sliderobes Experienced in Web Analytics, User Researcher & A/B Testing
  3. 3. @matt_lacey12 #CampDigital
  4. 4. @matt_lacey12 #CampDigital What should we be testing? Pricing & Price Framing Proposition Testing Optimising for Engagement Optimising for Conversion TALK OVERVIEW
  5. 5. @matt_lacey12 #CampDigital WHAT SHOULD YOU BE TESTING?
  6. 6. @matt_lacey12 #CampDigital WHAT’S WRONG WITH TESTING BUTTON COLOURS Disclosure: We test buttons. Size, Prominence, Copy, Positioning and yes, Colour. But: There are likely to be constraints that limit the number of tests that you can run based on design & developer resource In most situations we find that there are tests that can be run that will deliver more interesting results and more value
  7. 7. @matt_lacey12 #CampDigital SECTION TITLE THE CRO PROCESS
  8. 8. @matt_lacey12 #CampDigital SECTION TITLE WHAT DOES IT INVOLVE? Analytics User Research Cross Device User Testing Hypothesis Development Concept Development Psychology & Persuasion techniques Usability Best Practice Testing Strategy and Planning Test Setup & QA Test Results Analysis
  9. 9. @matt_lacey12 #CampDigital What is the data telling me about visitor behaviour? • Top landing pages • Pages with high traffic • Key leak or conversion points What are users telling me about their experiences? What can we observe about their behaviour? • Usability errors • Influential proposition messaging What does the business know about customer struggles or business challenges? • Customer service insights • Merchandising team HOW TO CHOOSE WHAT TO TEST
  10. 10. @matt_lacey12 #CampDigital Triage • Just Do It – JDI • Single Feature Test • Batch Testing • Radical Redesign Testing • Larger Redesign projects Triangulation • Multiple research methods confirm or strengthen the same issues Potential Impact vs Effort Required • Likelihood to impact key metrics vs • Design effort • Development effort • Development schedule HOW TO PRIORITISE YOUR TESTS
  11. 11. @matt_lacey12 #CampDigital 1. PRICE & PRICE FRAMING TESTS
  12. 12. @matt_lacey12 #CampDigital Through moderating or observing hundreds of user testing sessions we have often asked: WHY TEST PRICING? “If you could buy the same product from two different websites, what factors would be important in helping you choose which to buy from?” User consistently respond with: 1. Price 2. Delivery (price and timescales)
  13. 13. @matt_lacey12 #CampDigital 1.  Pricing should be based on value, not cost 2.  Prices should be tangible 3.  Prices should be comparable 4.  To change your prices, you must reframe 5.  Price differentiation is the key enabler of profit 6.  Pricing communication shapes the customers perception of value 7.  You must be prepared to lose some sales. Source: Leigh Caldwell:The Psychology of Price: How to use price to increase demand, profit and customer satisfaction SEVEN PRINCIPLES OF PRICING
  14. 14. @matt_lacey12 #CampDigital Three Principles of Anchoring 1.  The perception of price is often based on the context. 2.  Decision making and the power of suggestion 3.  Avoiding extremes Source: Price Intelligently http://blog.priceintelligently.com/blog/bid/181199/Price-Anchoring-to- Optimize-Your-Pricing-Strategy TECHNIQUE: PRICE ANCHORING
  15. 15. @matt_lacey12 #CampDigital Four Options: 1. Control 2. Variation A – (SAVE £22) 3. Variation B – (SAVE 38%) 4. Variation C – remove ‘(cancel anytime)’ from monthly WHICH TEST WON? 12.5%overall increase in subscriptions for Variation A (SAVE £22)
  16. 16. @matt_lacey12 #CampDigital 1.  Control - Annual £39.99 (Save £44) 2. Variation – £3.33 per month (£39.99 annually) WHICH TEST WON? 77%overall increase in annual subscriptions for the Variation
  17. 17. @matt_lacey12 #CampDigital Consider the range of price options available and test • This could apply to product or service costs, delivery costs, etc Run tests to make price more transparent • What do people get for their money • Test messaging that explains what people are paying for Control comparisons • Test using competitor prices for comparisons PRICE & PRICE FRAMING TAKEAWAYS
  18. 18. @matt_lacey12 #CampDigital 2. PROPOSITION TESTING
  19. 19. @matt_lacey12 #CampDigital Watch clip: http://www.youtube.com/watch?feature=player_embedded&v=ps7bB9-ksMY “ The second a visitor comes to you they have in mind: “Why should I buy from you versus all your competitors?” You need to display clear unique value propositions or unique campaign propositions. Everyone who is successful in online conversion is doing this. If you’re not, you’re wasting your time. Bryan Eisenberg, Godfather of CRO
  20. 20. @matt_lacey12 #CampDigital We can define this as 'Why should I buy from you'. Typically this is a short statement that is displayed prominently on the site. Here are a few examples: • ASOS: "Discover fashion online” • Naked Wines: "We are a customer funded wines business" UNIQUE VALUE PROPOSITION (UVP)
  21. 21. @matt_lacey12 #CampDigital GLOBAL HEADER UVP
  22. 22. @matt_lacey12 #CampDigital USPs - Unique Selling Proposition USPs are more focused on the shopping experience and what advantages visitors gain from shopping with you. Examples: • Free next day delivery over £50 • 365 Day Returns • The Official Online Store • The UK's largest stockist UNIQUE SELLING PROPOSITIONS (USPS)
  23. 23. @matt_lacey12 #CampDigital PRODUCT PAGE USP
  24. 24. @matt_lacey12 #CampDigital BASKET PAGE USPS
  25. 25. @matt_lacey12 #CampDigital Unique Campaign Propositions - UCP’s are messages based on specific campaigns or promotions. These messages will often be time limited. Examples: • Last Christmas Deliveries on 18th Dec • XX% off this weekend only • Free next day delivery all week UNIQUE CAMPAIGN PROPOSITIONS
  26. 26. @matt_lacey12 #CampDigital UNIQUE CAMPAIGN PROPOSITIONS
  27. 27. @matt_lacey12 #CampDigital THE NORTH FACE USP BAR TEST
  28. 28. @matt_lacey12 #CampDigital BEFORE AFTER 4.35% increase in CR This equated to a six-figure increase in revenue.
  29. 29. @matt_lacey12 #CampDigital GET REVISING SIGN-UP TABLE TEST
  30. 30. @matt_lacey12 #CampDigital GET REVISING SIGN-UP TABLE “ Price doesn’t put me off, although the basic package is more than enough for me. I think the site has already got everything I need. Moderated User testing participants
  31. 31. @matt_lacey12 #CampDigital
  32. 32. @matt_lacey12 #CampDigital GET REVISING SIGN-UP TABLE 185%increase in premium subscriptions with a small drop in free subscription. This was seen to be a good trade-off. You have more features held back to encourage people to subscribe at a later date.
  33. 33. @matt_lacey12 #CampDigital Test what features/functionality is given for free and what is paid for Test how your proposition is displayed/ framed on your site Test offering different thresholds for free delivery Test introducing urgency to the process PROPOSITION TESTING TAKEAWAYS
  34. 34. @matt_lacey12 #CampDigital 3. RADICAL REDESIGNS OPTIMISING FOR ENGAGEMENT
  35. 35. @matt_lacey12 #CampDigital This opens up the top of the conversion funnel. This is likely to be the point where you have the largest number of potential test participants. Search for landing or key site pages with high bounce rates or those that rarely lead to conversions Key Areas to Test • Homepage • Top landing pages • Brand sites/Brand content OPTIMISING FOR ENGAGEMENT
  36. 36. @matt_lacey12 #CampDigital RUSH HAIR HOMEPAGE TEST
  37. 37. @matt_lacey12 #CampDigital BEFORE AFTER 10.2%performance improvement in booking conversion rate
  38. 38. @matt_lacey12 #CampDigital MARKED BY TEACHERS ESSAY DETAIL PAGE
  39. 39. @matt_lacey12 #CampDigital BEFORE AFTER 10% improvement in CR
  40. 40. @matt_lacey12 #CampDigital BEFORE AFTER 6.8% improvement in CR
  41. 41. @matt_lacey12 #CampDigital Let users know that they have landed in the right place Make sure that pages have a clear focus and visual hierarchy. Users need to know where to begin. Test both simplifying the experience and adding in more detail A smaller amount of highly qualified visitors can trump large amounts of un- qualified visitors and convert better ENGAGEMENT OPTIMISATION TAKEAWAYS
  42. 42. @matt_lacey12 #CampDigital 4. RADICAL REDESIGNS OPTIMISING FOR CONVERSION
  43. 43. @matt_lacey12 #CampDigital Testing towards the end of your funnel helps to convert users who are closest to converting and tend to have higher intent. Improvements here opens up the bottom of the conversion funnel. Simplifying and removing issues that slow momentum is crucial. Key Areas to Test • Checkout (login, delivery, payment) • Contact/Sign-up forms OPTIMISING FOR CONVERSIONS
  44. 44. @matt_lacey12 #CampDigital THE STUDENT ROOM HOMEPAGE TEST
  45. 45. @matt_lacey12 #CampDigital BEFORE AFTER 72% improvement in Registrations from the homepage
  46. 46. @matt_lacey12 #CampDigital BEFORE AFTER 7.4% CR improvement from sign-up page
  47. 47. @matt_lacey12 #CampDigital Don’t be afraid to offer visitors an opportunity to convert at an early stage Use persuasive techniques to pull users through (not push!) Simplify forms to the bare minimum fields required Reduce errors. There’s a huge correlation between users getting an error and abandoning the funnel Provide key supporting proposition messaging throughout the process Re-enforce positive messages (e.g. security in checkout) CONVERSION OPTIMISATION TAKEAWAYS
  48. 48. @matt_lacey12 #CampDigital With the right kind of research and preparation the odds are stacked in your favour. Therefore: Be data driven. Be bold. Medium risk, high reward. FINAL TAKEAWAY
  49. 49. @matt_lacey12 #CampDigital Conversion optimisation case studies Usability best practice User research insights CRO presentation slides Persuasive design slides Research & optimisation tools One final resources link... bit.ly/CROresources
  50. 50. @matt_lacey12 #CampDigital Are there any questions? THANKS FOR LISTENING MATT LACEY Head of Optimisation @matt_lacey12

×