Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Realise the Benefits of Holistic Testing

Delivered by Kath Pay, CEO of Holistic Email Marketing, at Figaro Digital's Email Marketing & CRM Seminar on Thursday 9 February 2017.

Aimed at marketing professionals looking to push the power of their email marketing, this session reveals how to leverage the testing ability of your emails to improve the performance of your other marketing acquisition and conversion channels. You will learn how to build a hypothesis into your emails, to drive the actions that provide the answers you're looking for. As well as how to use a push channel such as email to inform and improve pull channel performance. Plus, tactics to test for long-term results as well as short-term results.

  • Be the first to comment

Realise the Benefits of Holistic Testing

  1. 1. Realise the Benefits of Holistic Testing Kath Pay, Holistic Email Marketing
  2. 2. Kath Pay, Founder & CEO Holistic Email Marketing @kathpay
  3. 3. US Interactive Marketing Spend 2016 Orders by Marketing Channel Forrester Custora
  4. 4. Using the actual numbers, let’s look at them in some pretty graphs Spend Social 6.52% Email 3.22% Display 36.02% Search 43.49% Orders Social 1.50% Email 18.40% Display 1.10% Search 41.80% Forester Custora
  5. 5. Difference Email 15.18% Search -1.69% Social -5.02% Display -34.92%
  6. 6. And that’s with the attribution being inaccurate! Source: DMA UK 57% save the email for later and 48% bear the information in mind for later. 47% go to the website via another route
  7. 7. What if? • Image of girl with ???? Or person with surprised (happy) face
  8. 8. Why test? You want to find out what resonates with your prospects and customers, so they’re more likely to action your objective. Simple!
  9. 9. Adhoc testing Definition: Random, occasional test of a factor with the aim of determining which version delivers the best result. No methodology or hypothesis is used in this process.
  10. 10. Let’s test the subject line! Control: Our sale is on now! Get 15% off today! Variant: Our sale is on today! Get 15% off now!
  11. 11. Holistic testing Definition: This is a regularly-performed test based on a hypothesis, with the two-fold aim of not only finding out which version delivers the best result, but also aims to discover insights about the audience.
  12. 12. What makes my audience tick? Hypothesis: Loss aversion copy is a stronger motivator towards conversions than benefit-led copy because people hate losing out more than they enjoy benefiting. Control: Our sale is on now! Get 15% off today! Variant: 15% off today only. Don’t miss out!
  13. 13. Who makes up your email database? Your Target Market!
  14. 14. Your database is your target market! Use #email as your primary testing tool & roll out findings to other channels @kathpay @Figaro_Digital Tweet this
  15. 15. Identify your objective1
  16. 16. Adhoc Testing Objectives 1. Gain an immediate uplift in an email campaign Holistic Testing Objectives 1. Gain an immediate uplift in conversions of an email campaign 2. Gain insights into your customers that can be used within your other channels & increase revenue across the board 3. Make the email channel the main channel for testing & bring additional budget & resources to it
  17. 17. Have a desire to learn about your audience 2
  18. 18. Test strategic issues such as a pain- point or business challenge. Ask yourself “What makes my audience tick?” @kathpay @Figaro_Digital Tweet this
  19. 19. Identify what question are you trying to answer? 3
  20. 20. Create a Hypothesis4
  21. 21. It needs to centre around 3 main things 1. Presumed problem 2. Proposed solution 3. Anticipated result (and reason why)
  22. 22. A Hypothesis can be proved or disproved A hypothesis pretty much says, “I think by making this change, it will cause this effect.” So, based on your results, you should be able to say “this is true” or “this is false.”…. BECAUSE.......
  23. 23. Let’s look at some hypotheses to use in email Hypothesis Factor tested Channels to roll it out to Emotional question will generate more sales than a directive statement Subject line, CTA, title, copy Website, landing page, banner ads, PPC ads, social media, retargeting ads Double Loyalty points will generate more sales than 15% loyalty points Subject line, CTA, title, copy Website, landing page, retargeting ads, social media Emotive image of person smiling & wearing outfit will generate more sales than image displaying outfit laid out. image Website, landing page, banner ads, Social media, retargeting ads
  24. 24. Select the factor to test5
  25. 25. When you use a hypothesis to test #email you’re not restricted to testing the same element. The hypothesis stands @kathpay @Figaro_Digital Tweet this
  26. 26. Identify your success metric6
  27. 27. Focus on & measure the correct metric! A: Increased total orders by 21% And revenue by 35%, even though A had a 10.5% lower open rate than B. Sent 50/50 split test over 4 weeks to abandoners Source: Native Remedies A Reminder: Items Saved in Your Cart – Shop Now for an Extra 10% off B Come back and Save an Extra 10%
  28. 28. Analyse and learn7
  29. 29. Create a test plan
  30. 30. Metrics & Fields to include • Total Sent • Open Rate • Click to Open Rate • Products purchased • Conversion Rate • AOV • Statistical Confidence • Conclusion • Recommendation
  31. 31. Statistical Validity
  32. 32. Test hypothesis multiple times • Because you’re performing hypothesis-led tests you can: • Use different wording • Potentially use multiple factors
  33. 33. Roll results out to other channels8
  34. 34. Perform tests in email
  35. 35. Hypothesis: Informational subject line will deliver more conversions because a mortgage is a serious and lengthy decision-making process Urgency Informational Winner! 49.2% uplift 99% confidence
  36. 36. Winner! 63% uplift 95% confidence Hypothesis: Video content will deliver better results because video content is very consumable Video ContentDatasheet Content
  37. 37. Roll these insights out to other channels & continually optimise
  38. 38. Aggregation of marginal gains The BG cycling team’s winning philosophy is ‘aggregation of marginal gains’: Get a lot of small things right, put them together, and they add up to a significant gain,27290,17547_5792058,00.html “It means taking the 1% from everything you do; finding a 1% margin for improvement in everything you do.”
  39. 39. Bring more ££ & value into email marketing!9
  40. 40. Tests which would have been quicker, cheaper and easier to test in email
  41. 41. Winner! Hypothesis: “Get A Quote” would perform best. Because the water treatment services are expensive, the idea of a “quote” would establish more trust – as it suggests a personal experience with a representative, allowing you to learn more about the company’s water treatment options. Non-personalPersonal 104% uplift 96% confidence
  42. 42. Winner! Hypothesis: Shortening the price would indicate the item costs less. So, visitors would be most receptive to the truncated price, without the zeros LongShort 9.3% increase in add-to-cart clicks 29% rise in visits/order 47% lift in RPV 99% confidence
  43. 43. It’s a cycle of continuous learning Identify your objective Have a desire to know your customers Identify a question or pain Create your hypothesis Select the factor to test Test in email Identify your success metric Analyze & Learn Apply to appropriate channels Bring value & ££ into email
  44. 44. @holisticemail