ADMA Video Testing and Measurement


Published on

The presentation discusses the significance of testing in marketing campaigns.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

ADMA Video Testing and Measurement

  1. 1. TestingAustralian Direct Marketing Association
  2. 2. What is testing? Testing is a marketing experiment in a live environment: •  Observing what customers actually do rather than what they say they are going to do •  Monitoring how results vary as various components of a campaign are altered. •  Using the results of testing to: –  predict the outcome of future activity. –  Improve our marketing performance Australian Direct Marketing Association
  3. 3. 3 approaches 1.  Test on a small scale Roll-out first then roll-out Test 2. Do testing on a portion Live campaign of a live campaign Test 3. Same approach to all Live campaign ? then analyse results among different segments to spot variations Australian Direct Marketing Association
  4. 4. Summary of testing optionsTest the important things…..Message components Delivery components•  Product •  Targeting & segmentation•  Proposition •  Communication channels•  Offer •  Format•  Creative •  Timing•  Call to action Australian Direct Marketing Association
  5. 5. Testing case study •  Two simple objectives –  Improve response rates –  Increase amount donated •  Understanding donor segments –  Relationship to disease –  ValueWith permission from Salmat Australian Direct Marketing Association
  6. 6. Test different propositions for each segment •  Relationship to disease –  Have the disease –  Parent of someone with the disease –  Relative / friend of someone with the disease –  No relationship to the diseaseWith permission from Salmat Australian Direct Marketing Association
  7. 7. Personalisation case study •  Value –  Variable donations boxes based on last donation, increased in increments of 20%With permission from Salmat Australian Direct Marketing Association
  8. 8. Testing case studyWith permission from Salmat Australian Direct Marketing Association
  9. 9. Example: Landing Page Test Options Page components 1. Headline 2. Secondary Headline 3 Body Copy 4 Social Proof 5/6 Call to Action 7  Navigation 8  Image 9  Above/below the Fold Australian Direct Marketing Association
  10. 10. Guidelines for accurate testing• Ensure you always have a “control” – So you have a benchmark for comparison• Have a big enough sample to test with – To produce reliable results• Ensure your test sample is not skewed at all (compared to control) – Use a randomly-selected sample• Decide how you are going to measure performance: –The measurement chosen must prove/disprove your hypothesis Australian Direct Marketing Association
  11. 11. The importance of a control Offer test Here there is no control: Response rate - A separate offer has been run in each month New offer A Standard offer New offer B - Offer A appears to have out- performed the current offer - Offer B appears to have performed worse = Offer A appears to win May June July Australian Direct Marketing Association
  12. 12. The importance of a control Offer test Introduction of control: Response rate - The current offer has been run in each month as a New offer A benchmark Standard offer New offer B - Offer A did not perform as well as the current offer - Offer B performed better than the current offer May June July = Offer B is the real winner Australian Direct Marketing Association
  13. 13. The “significance” of a test resultResponse rate The test out-performed the control by 0.3% 1.7% 1.4% Should you adopt New Offer A – or could it be just a chance result? We need to know whether these results are likely to be repeated if we ran the test again Control New Offer A Australian Direct Marketing Association
  14. 14. Having a Big Enough Sample• In direct marketing, your sample is Required Sample Quantity typically a subset of your customer base or contact list•  In digital, your sample is typically the volume of unique visitors or unique impressions accumulated over a period of time•  You need a quantity of sample that reflects your desire to detect even 50%$ 60%$ 70%$ 80%$ 90%$ 95%$ 99%$ small differences between your test and control groups Ability to Detect Statistically Significant Differences Australian Direct Marketing Association
  15. 15. Not Just Statistical SignificanceDo a sense-check when interpreting results:• What was the competition doing when this test was running?• Just because this worked in one location does it mean it will work in another?• The offer was successful in Summer – would it still work in Winter?•  Were there any other abnormal factors in the marketplace which might have affected the response? Australian Direct Marketing Association
  16. 16. Not Just Statistical SignificanceDid we make money? When we balance the costs of a campaign against the incremental volume of sales – was it worth it?What should we test next? e.g. offer A performed better than offer B, what will beat offer A? Australian Direct Marketing Association
  17. 17. MeasurementAustralian Direct Marketing Association
  18. 18. Tracking the customer journeyA buyer typicallygoes through severalstages before they buy Awareness Interest Desire Action…which means we should havemeasures to trackperformance for each stage Reach Influence Closure Australian Direct Marketing Association
  19. 19. Example: eCommerce Awareness Daily  Unique  Visitors   Interest Product  Page  Views   Desire Completes  a  Registra:on  Form   Action Purchase   Australian Direct Marketing Association
  20. 20. Tracking issues How do you match response to source when: –  Your campaign runs in several channels simultaneously –  Your product is sold through a third party sales channel –  Your product has a long consideration period Australian Direct Marketing Association
  21. 21. Common tracking optionsOutbound Channel Response Channel Offline Unique campaign and Call-Centre comms channel Broadcast specific phone numbers Mail Unique redemption In-Store codes Display Media Specific Search Call to Website Action Australian Direct Marketing Association
  22. 22. Indirect sales – a missing link Browse Known Clicks/visits Buy Unknown Sales Enquire Known Calls Australian Direct Marketing Association
  23. 23. Long sales cycles Sales Time What does this response curve look like for your product? •  Track enquiries & sales over a long period •  Establish a pattern •  Create a rule of thumb Australian Direct Marketing Association
  24. 24. Rule of Thumb Approach•  Can be used for indirect sales as well as an ‘early read’ for long campaign cycles•  Typical approach: 1.  Establish a ratio for website visits or calls to reseller enquiries/ sales 2.  Establish a pre-campaign baseline for calls and website visits 3.  Measure the uplift in calls/visits during and following the promotion 4.  Extrapolate to sales using typical ratio Australian Direct Marketing Association