Your SlideShare is downloading. ×
0
TestingAustralian Direct Marketing Association
What is testing? Testing is a marketing experiment in a live environment: •  Observing what customers actually do rather t...
3 approaches                                                   1.  Test on a small scale                                  ...
Summary of testing optionsTest the important things…..Message components                          Delivery components•  Pr...
Testing case study               •  Two simple objectives                       –  Improve response rates                 ...
Test different propositions for each segment               •  Relationship to disease                        –     Have th...
Personalisation case study             •  Value                      –  Variable donations boxes based on last donation, i...
Testing case studyWith permission from Salmat                 Australian Direct Marketing Association
Example: Landing Page Test Options                                           Page components                              ...
Guidelines for accurate testing• Ensure you always have a “control”  – So you have a benchmark for comparison• Have a big ...
The importance of a control               Offer test                            Here there is no control: Response   rate ...
The importance of a control              Offer test                             Introduction of control: Response   rate  ...
The “significance” of a test resultResponse  rate                                        The test out-performed the contro...
Having a Big Enough Sample• In direct marketing, your sample is                                              Required Samp...
Not Just Statistical SignificanceDo a sense-check when interpreting results:• What was the competition doing when this tes...
Not Just Statistical SignificanceDid we make money?   When we balance the costs of a campaign against the   incremental vo...
MeasurementAustralian Direct Marketing Association
Tracking the customer journeyA buyer typicallygoes through severalstages before they buy                    Awareness     ...
Example: eCommerce Awareness                                       Daily	  Unique	  Visitors                              ...
Tracking issues How do you match response to source when:   –  Your campaign runs in several channels      simultaneously ...
Common tracking optionsOutbound Channel                            Response Channel     Offline                           ...
Indirect sales – a missing link                          Browse                            Known                          ...
Long sales cycles        Sales                                                 Time What does this response curve look lik...
Rule of Thumb Approach•  Can be used for indirect sales as well as an ‘early read’ for long   campaign cycles•  Typical ap...
Upcoming SlideShare
Loading in...5
×

ADMA Video Testing and Measurement

237

Published on

The presentation discusses the significance of testing in marketing campaigns.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
237
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "ADMA Video Testing and Measurement"

  1. 1. TestingAustralian Direct Marketing Association
  2. 2. What is testing? Testing is a marketing experiment in a live environment: •  Observing what customers actually do rather than what they say they are going to do •  Monitoring how results vary as various components of a campaign are altered. •  Using the results of testing to: –  predict the outcome of future activity. –  Improve our marketing performance Australian Direct Marketing Association
  3. 3. 3 approaches 1.  Test on a small scale Roll-out first then roll-out Test 2. Do testing on a portion Live campaign of a live campaign Test 3. Same approach to all Live campaign ? then analyse results among different segments to spot variations Australian Direct Marketing Association
  4. 4. Summary of testing optionsTest the important things…..Message components Delivery components•  Product •  Targeting & segmentation•  Proposition •  Communication channels•  Offer •  Format•  Creative •  Timing•  Call to action Australian Direct Marketing Association
  5. 5. Testing case study •  Two simple objectives –  Improve response rates –  Increase amount donated •  Understanding donor segments –  Relationship to disease –  ValueWith permission from Salmat Australian Direct Marketing Association
  6. 6. Test different propositions for each segment •  Relationship to disease –  Have the disease –  Parent of someone with the disease –  Relative / friend of someone with the disease –  No relationship to the diseaseWith permission from Salmat Australian Direct Marketing Association
  7. 7. Personalisation case study •  Value –  Variable donations boxes based on last donation, increased in increments of 20%With permission from Salmat Australian Direct Marketing Association
  8. 8. Testing case studyWith permission from Salmat Australian Direct Marketing Association
  9. 9. Example: Landing Page Test Options Page components 1. Headline 2. Secondary Headline 3 Body Copy 4 Social Proof 5/6 Call to Action 7  Navigation 8  Image 9  Above/below the Fold Australian Direct Marketing Association
  10. 10. Guidelines for accurate testing• Ensure you always have a “control” – So you have a benchmark for comparison• Have a big enough sample to test with – To produce reliable results• Ensure your test sample is not skewed at all (compared to control) – Use a randomly-selected sample• Decide how you are going to measure performance: –The measurement chosen must prove/disprove your hypothesis Australian Direct Marketing Association
  11. 11. The importance of a control Offer test Here there is no control: Response rate - A separate offer has been run in each month New offer A Standard offer New offer B - Offer A appears to have out- performed the current offer - Offer B appears to have performed worse = Offer A appears to win May June July Australian Direct Marketing Association
  12. 12. The importance of a control Offer test Introduction of control: Response rate - The current offer has been run in each month as a New offer A benchmark Standard offer New offer B - Offer A did not perform as well as the current offer - Offer B performed better than the current offer May June July = Offer B is the real winner Australian Direct Marketing Association
  13. 13. The “significance” of a test resultResponse rate The test out-performed the control by 0.3% 1.7% 1.4% Should you adopt New Offer A – or could it be just a chance result? We need to know whether these results are likely to be repeated if we ran the test again Control New Offer A Australian Direct Marketing Association
  14. 14. Having a Big Enough Sample• In direct marketing, your sample is Required Sample Quantity typically a subset of your customer base or contact list•  In digital, your sample is typically the volume of unique visitors or unique impressions accumulated over a period of time•  You need a quantity of sample that reflects your desire to detect even 50%$ 60%$ 70%$ 80%$ 90%$ 95%$ 99%$ small differences between your test and control groups Ability to Detect Statistically Significant Differences Australian Direct Marketing Association
  15. 15. Not Just Statistical SignificanceDo a sense-check when interpreting results:• What was the competition doing when this test was running?• Just because this worked in one location does it mean it will work in another?• The offer was successful in Summer – would it still work in Winter?•  Were there any other abnormal factors in the marketplace which might have affected the response? Australian Direct Marketing Association
  16. 16. Not Just Statistical SignificanceDid we make money? When we balance the costs of a campaign against the incremental volume of sales – was it worth it?What should we test next? e.g. offer A performed better than offer B, what will beat offer A? Australian Direct Marketing Association
  17. 17. MeasurementAustralian Direct Marketing Association
  18. 18. Tracking the customer journeyA buyer typicallygoes through severalstages before they buy Awareness Interest Desire Action…which means we should havemeasures to trackperformance for each stage Reach Influence Closure Australian Direct Marketing Association
  19. 19. Example: eCommerce Awareness Daily  Unique  Visitors   Interest Product  Page  Views   Desire Completes  a  Registra:on  Form   Action Purchase   Australian Direct Marketing Association
  20. 20. Tracking issues How do you match response to source when: –  Your campaign runs in several channels simultaneously –  Your product is sold through a third party sales channel –  Your product has a long consideration period Australian Direct Marketing Association
  21. 21. Common tracking optionsOutbound Channel Response Channel Offline Unique campaign and Call-Centre comms channel Broadcast specific phone numbers Mail Unique redemption In-Store codes Display Media Specific Search Call to Website Action Australian Direct Marketing Association
  22. 22. Indirect sales – a missing link Browse Known Clicks/visits Buy Unknown Sales Enquire Known Calls Australian Direct Marketing Association
  23. 23. Long sales cycles Sales Time What does this response curve look like for your product? •  Track enquiries & sales over a long period •  Establish a pattern •  Create a rule of thumb Australian Direct Marketing Association
  24. 24. Rule of Thumb Approach•  Can be used for indirect sales as well as an ‘early read’ for long campaign cycles•  Typical approach: 1.  Establish a ratio for website visits or calls to reseller enquiries/ sales 2.  Establish a pre-campaign baseline for calls and website visits 3.  Measure the uplift in calls/visits during and following the promotion 4.  Extrapolate to sales using typical ratio Australian Direct Marketing Association
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×