• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
2013 Salsa Community Conference - A/B Testing 101
 

2013 Salsa Community Conference - A/B Testing 101

on

  • 380 views

 

Statistics

Views

Total Views
380
Views on SlideShare
380
Embed Views
0

Actions

Likes
0
Downloads
6
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    2013 Salsa Community Conference - A/B Testing 101 2013 Salsa Community Conference - A/B Testing 101 Presentation Transcript

    • A/B TESTING 101
    • Agenda • • • • • • What is an A/B test? What can we test (and find out)? How do we set up an A/B test in Salsa? How do we find out the ‘winner’? What should we be careful with? Q&A
    • What is an A/B test? • • • • STOP - concept time! An A/B test splits off a percentage of your recipient group for test A, and another percentage for test B. One of the tests is usually a 'control' - that is, unchanged from the original subject/content. The other is modified to see if the response is stronger than the control. Based on your chosen metrics, you may choose to change your email blast, or leave it alone, before sending to the remainder of your list.
    • What are we testing? • Quick and Easy: Subject Line From Info Time of Day Day of the Week A bit more involved: Body Copy Layout and Design Call to Action type Metrics to Track: Opens Click-throughs Completions Unsubs
    • How do we set it up in Salsa? Step 7: Summary page • Test Blasts section - Create test blasts button - by default creates 2 tests. • Set the targeting sliders: • • Typically, you'll se a 10% test, with 5% to each test After creating your tests, make your changes. As mentioned, it’s best to keep a control blast which is unedited.
    • How do we set it up in Salsa? Step 7: Summary page • Quick edit – change the targeting percentage, reference name, or subject - OR click 'edit full email blast' to change content. • Send or schedule your tests at the same time you plan to send your main blast. • After sending, let it run its course – don’t pull the trigger too early, but don’t wait forever either. • After determining which one has won, you'll either choose "Leave as is" if the control blast won; otherwise choose the blast to replace with.
    • How do we tell which won? • A/B test significance calculators • • • • • • http://getdatadriven.com/ab-significance-test http://www.usereffect.com/split-test-calculator http://super.hubspot.com/calculator/ There are a hundred points I could make here about statistics, that I’m not qualified or mathematical enough to be making. But keep an eye on unsubs too! Do a debrief after your campaign, and keep tracking! • • Should you switch tactics permanently? Should you re-test soon?
    • What should we be careful with? • • • • • NEVER TEST MORE THAN ONE VARIABLE AT A TIME. Timing - send your main blast at the same time you send your tests to remove that variable. Reduce other variables as possible. General problems of statistical significance, bias, etc. Just don’t overdo it.
    • THANK YOU!