Why Guess When You Can Test? Testing Techniques from the Experts


Published on

The fact is that most of us are still sending emails where we're guessing if we are going to be successful. Why guess? Test! Join this webinar as we cover a variety of testing techniques and tools to allow you to know before the email goes whether your subject line is appealing, you've chosen the right call-to-action and how the email will look on every device known to Mankind. We'll teach you about the latest tools for testing, how to correctly and easily perform A/B split testing and how to troubleshoot your emails before you send.

Published in: Marketing, Technology, Business
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Why Guess When You Can Test? Testing Techniques from the Experts

  1. 1. TheU: Testing…Testing…123
  2. 2. Your Presenter: www.highroadsolution.com Jenny Lassi has been affiliated with HighRoad Solution for 8 years first as a client and then as an employee.With experience handling deployments and deliverability for a wide range of clients with optimized recipient engagement top- of-mind, Jenny routinely trouble shoots deliverability issues from ISP level filtering to email client level filtering (email Forensics as she refers to them). She also consults with clients on authentication, content, content strategy, email preference center management & other deliverability optimization tactics.
  3. 3. Today’s Content www.highroadsolution.com • What is AB Testing? • What is Multi-VariatTesting? • The 5 C’s & How They Apply to Testing • Starting Simply • Testing the Complex • Best practices • Resources takeaways
  4. 4. What is AB Testing? www.highroadsolution.com
  5. 5. AB Testing Defined www.highroadsolution.com A/B testing is a term commonly used in web development, online marketing, and other forms of advertising to describe simple randomized experiments with two variants,A and B, which are the control and treatment in the controlled experiment. The formal or scientific name used for this process and other related processes is hypothesis testing. Other names include randomized controlled experiments, online controlled experiments, and split testing.
  6. 6. What is Multivariate Testing? www.highroadsolution.com
  7. 7. Multivariate Testing Defined www.highroadsolution.com In internet marketing, multivariate testing is a process by which more than one component of a website may be tested in a live environment. It can be thought of in simple terms as numerous A/B tests performed on one page at the same time.A/B tests are usually performed to determine the better of two content variations; multivariate testing can theoretically test the effectiveness of limitless combinations. Note: Be cautious of testing too many variables as it can be difficult to understand what is behind the lift in response.
  8. 8. eMail Strategy www.highroadsolution.com
  9. 9. The 5 C’s of Email Marketing Strategy www.highroadsolution.com •Configuration (Clean House!) •Content •Create •Criticize •Continue
  10. 10. The 5 C’s: Configuration • IP Address (Dedicated or Shared) • DKIM (Domain Keys) • Sender ID (SPF Record) www.highroadsolution.com
  11. 11. The HighRoad Solution Deliverability Cops – Dedicated IP www.highroadsolution.com • Sender Rejected • Message Filtered • Unknown Users • Infrastructure • Volume • SpamTraps • Blacklists • Complaints • FBL Compaints (Routed to your Opt-Outs)
  12. 12. The HighRoad Solution Deliverability Cops – Shared IP www.highroadsolution.com • Spam Traps • Blacklists • Complaints
  13. 13. Authentication www.highroadsolution.com High Level View: DKIM/Sender ID (it’s not a scary term or complicated at all… I promise) • You tell us what domain(s) you send from • We supply you with 4 snippets of text that you add to your domain’s DNS and instructions • You add the txt records to your DNS and tell us when you’re done • We test/validate the keys and enable on your account
  14. 14. Why not just blast? www.highroadsolution.com Invalid email Addresses + Excessive email Bounces + Emails Filtered to Junk/SPAM Folders = Wasted $$
  15. 15. eMail Filtering www.highroadsolution.com Email Filtering Layers
  16. 16. Do you know if your eMails are reaching an inbox? www.highroadsolution.com When a SureSend Domain Delivery Report is enabled when sending a Live message, you will see the Domain Delivery tab on the email stats
  17. 17. Do you know if your eMails are reaching an inbox? www.highroadsolution.com
  18. 18. Cleaning House www.highroadsolution.com Do whatever is in your control to reach an inbox! • Data Hygiene (Invalids/Bounces) • DKIM/Sender ID Authentication • Address any issues with filtering that you can THEN… and only then should you AB Test
  19. 19. The 5 C’s: Content Common ContentTests: • Subject Lines • Lead Story • TOC • More text vs. read more www.highroadsolution.com
  20. 20. The 5 C’s: Create • Test standard html vs. responsive template • Call-To-Action Button Color & Placement • Layout • Images www.highroadsolution.com
  21. 21. Case Study Workshop Can you trust your gut? www.highroadsolution.com
  22. 22. Case Study Workshop Can being more verbose get a higher response? www.highroadsolution.com
  23. 23. www.highroadsolution.com Version A
  24. 24. www.highroadsolution.com Version B
  25. 25. Case Study Workshop What do you think? Version A? Version B? www.highroadsolution.com
  26. 26. Case Study Workshop Version B won an astounding 247% more clicks www.highroadsolution.com
  27. 27. Case Study Workshop Does a larger lead story get people to click? www.highroadsolution.com
  28. 28. www.highroadsolution.com Version A
  29. 29. www.highroadsolution.com Version B
  30. 30. Case Study Workshop What do you think? Version A? Version B? www.highroadsolution.com
  31. 31. Case Study Workshop Version B, the one without bigger featured content, increased clicks by 75% and online registrations by a stunning 400% www.highroadsolution.com
  32. 32. Case Study Workshop Does more or less white space increase engagement? www.highroadsolution.com
  33. 33. www.highroadsolution.com Version A
  34. 34. www.highroadsolution.com Version B
  35. 35. Case Study Workshop What do you think? Version A? Version B? www.highroadsolution.com
  36. 36. Case Study Workshop Version A Increased clickthroughs by 5.6%, and heatmaps showed that there was a higher engagement as a whole across the entire email. www.highroadsolution.com
  37. 37. Case Study Workshop Subject line straight shooter or a bit playful? www.highroadsolution.com
  38. 38. www.highroadsolution.com Version A
  39. 39. www.highroadsolution.com Version B
  40. 40. Case Study Workshop What do you think? Version A? Version B? www.highroadsolution.com
  41. 41. Case Study Workshop Version B – the playful version – increased opens for French-speaking Canadians. However, Version A – the plainspoken version – increased opens for English-speaking Canadians. www.highroadsolution.com
  42. 42. Case Study Workshop Which local time works better for B2B email offers 9am or 1pm? www.highroadsolution.com
  43. 43. www.highroadsolution.com Version A – 9am
  44. 44. www.highroadsolution.com Version B – 1pm
  45. 45. Case Study Workshop What do you think? Version A? Version B? www.highroadsolution.com
  46. 46. Case Study Workshop Version A, the email sent at 9 am local time at four U.S. time zones increased lifts in all time zones – here are the specifics: • EST – 12.5% lift in opens • CST – 100% lift in opens • MDT – 35% lift in opens • PST – 4,090% STUNNING lift in opens www.highroadsolution.com
  47. 47. The 5 C’s: Criticize • Analyze delivery metrics (bounces, opens & clicks) • Use bounce logs to address deliverability issues 24-48 hours post deployment (not a month later) • Isolate subscribers who are not engaged with your emails (no opens/click throughs) • Create an AB Test Win-Back campaign for non- engaged subscribers with an incentive to respond • Analyze how many subscribers opt-down or unsubscribe from particular messages (maybe the perceived value of content is low) www.highroadsolution.com
  48. 48. The 5 C’s: Continue Use what you have learned and apply it to future deployments www.highroadsolution.com
  49. 49. Best PracticesTakeaways www.highroadsolution.com • Be more strategic with your testing with the 5-C’s. • You can learn from other AB Tests, but testing against your own audience is the only way to see what increases engagement. • Send to known engaged subscribers when testing or your results may vary based on inbox placement.
  50. 50. Questions? www.highroadsolution.com
  51. 51. Shortcuts Control what you CAN control about deliverability: • DKIM/Sender ID Authentication • List Hygiene • Having an Email Preference Center • Deliver content members or registrants want, when they want it, how frequently they want it and looks good where they read it Analyze and learn from AB Split testing to adjust how you send future emails: • Use A/B split testing to learn about what has the best engagement • Review creative/content that had the best engagement and do more of that • Review bounce logs daily to see if any domains may be blocking you and reach out to postmaster@ and abuse@ those domains to ask them to whitelist you www.highroadsolution.com
  52. 52. Thank you! www.highroadsolution.com Other questions for Jenny? eMail = jenny@highroadsolution.com Twitter = @highroadjenny
  53. 53. References & Resources https://en.wikipedia.org/wiki/Ab_testing http://en.wikipedia.org/wiki/Multivariate_testing http://whichtestwon.com/ www.highroadsolution.com