Your SlideShare is downloading. ×
ADMA Video Testing and Measurement
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

ADMA Video Testing and Measurement


Published on

The presentation discusses the significance of testing in marketing campaigns.

The presentation discusses the significance of testing in marketing campaigns.

Published in: Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. TestingAustralian Direct Marketing Association
  • 2. What is testing? Testing is a marketing experiment in a live environment: •  Observing what customers actually do rather than what they say they are going to do •  Monitoring how results vary as various components of a campaign are altered. •  Using the results of testing to: –  predict the outcome of future activity. –  Improve our marketing performance Australian Direct Marketing Association
  • 3. 3 approaches 1.  Test on a small scale Roll-out first then roll-out Test 2. Do testing on a portion Live campaign of a live campaign Test 3. Same approach to all Live campaign ? then analyse results among different segments to spot variations Australian Direct Marketing Association
  • 4. Summary of testing optionsTest the important things…..Message components Delivery components•  Product •  Targeting & segmentation•  Proposition •  Communication channels•  Offer •  Format•  Creative •  Timing•  Call to action Australian Direct Marketing Association
  • 5. Testing case study •  Two simple objectives –  Improve response rates –  Increase amount donated •  Understanding donor segments –  Relationship to disease –  ValueWith permission from Salmat Australian Direct Marketing Association
  • 6. Test different propositions for each segment •  Relationship to disease –  Have the disease –  Parent of someone with the disease –  Relative / friend of someone with the disease –  No relationship to the diseaseWith permission from Salmat Australian Direct Marketing Association
  • 7. Personalisation case study •  Value –  Variable donations boxes based on last donation, increased in increments of 20%With permission from Salmat Australian Direct Marketing Association
  • 8. Testing case studyWith permission from Salmat Australian Direct Marketing Association
  • 9. Example: Landing Page Test Options Page components 1. Headline 2. Secondary Headline 3 Body Copy 4 Social Proof 5/6 Call to Action 7  Navigation 8  Image 9  Above/below the Fold Australian Direct Marketing Association
  • 10. Guidelines for accurate testing• Ensure you always have a “control” – So you have a benchmark for comparison• Have a big enough sample to test with – To produce reliable results• Ensure your test sample is not skewed at all (compared to control) – Use a randomly-selected sample• Decide how you are going to measure performance: –The measurement chosen must prove/disprove your hypothesis Australian Direct Marketing Association
  • 11. The importance of a control Offer test Here there is no control: Response rate - A separate offer has been run in each month New offer A Standard offer New offer B - Offer A appears to have out- performed the current offer - Offer B appears to have performed worse = Offer A appears to win May June July Australian Direct Marketing Association
  • 12. The importance of a control Offer test Introduction of control: Response rate - The current offer has been run in each month as a New offer A benchmark Standard offer New offer B - Offer A did not perform as well as the current offer - Offer B performed better than the current offer May June July = Offer B is the real winner Australian Direct Marketing Association
  • 13. The “significance” of a test resultResponse rate The test out-performed the control by 0.3% 1.7% 1.4% Should you adopt New Offer A – or could it be just a chance result? We need to know whether these results are likely to be repeated if we ran the test again Control New Offer A Australian Direct Marketing Association
  • 14. Having a Big Enough Sample• In direct marketing, your sample is Required Sample Quantity typically a subset of your customer base or contact list•  In digital, your sample is typically the volume of unique visitors or unique impressions accumulated over a period of time•  You need a quantity of sample that reflects your desire to detect even 50%$ 60%$ 70%$ 80%$ 90%$ 95%$ 99%$ small differences between your test and control groups Ability to Detect Statistically Significant Differences Australian Direct Marketing Association
  • 15. Not Just Statistical SignificanceDo a sense-check when interpreting results:• What was the competition doing when this test was running?• Just because this worked in one location does it mean it will work in another?• The offer was successful in Summer – would it still work in Winter?•  Were there any other abnormal factors in the marketplace which might have affected the response? Australian Direct Marketing Association
  • 16. Not Just Statistical SignificanceDid we make money? When we balance the costs of a campaign against the incremental volume of sales – was it worth it?What should we test next? e.g. offer A performed better than offer B, what will beat offer A? Australian Direct Marketing Association
  • 17. MeasurementAustralian Direct Marketing Association
  • 18. Tracking the customer journeyA buyer typicallygoes through severalstages before they buy Awareness Interest Desire Action…which means we should havemeasures to trackperformance for each stage Reach Influence Closure Australian Direct Marketing Association
  • 19. Example: eCommerce Awareness Daily  Unique  Visitors   Interest Product  Page  Views   Desire Completes  a  Registra:on  Form   Action Purchase   Australian Direct Marketing Association
  • 20. Tracking issues How do you match response to source when: –  Your campaign runs in several channels simultaneously –  Your product is sold through a third party sales channel –  Your product has a long consideration period Australian Direct Marketing Association
  • 21. Common tracking optionsOutbound Channel Response Channel Offline Unique campaign and Call-Centre comms channel Broadcast specific phone numbers Mail Unique redemption In-Store codes Display Media Specific Search Call to Website Action Australian Direct Marketing Association
  • 22. Indirect sales – a missing link Browse Known Clicks/visits Buy Unknown Sales Enquire Known Calls Australian Direct Marketing Association
  • 23. Long sales cycles Sales Time What does this response curve look like for your product? •  Track enquiries & sales over a long period •  Establish a pattern •  Create a rule of thumb Australian Direct Marketing Association
  • 24. Rule of Thumb Approach•  Can be used for indirect sales as well as an ‘early read’ for long campaign cycles•  Typical approach: 1.  Establish a ratio for website visits or calls to reseller enquiries/ sales 2.  Establish a pre-campaign baseline for calls and website visits 3.  Measure the uplift in calls/visits during and following the promotion 4.  Extrapolate to sales using typical ratio Australian Direct Marketing Association