Your SlideShare is downloading. ×
5 Steps for A/B Split Testing in Bronto
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

5 Steps for A/B Split Testing in Bronto


Published on

Learn how A/B split testing is a valuable instrument that all marketers should use to drive conversions. In this presentation you'll learn industry best practices, how to perform an A/B split test in …

Learn how A/B split testing is a valuable instrument that all marketers should use to drive conversions. In this presentation you'll learn industry best practices, how to perform an A/B split test in Bronto, and see testing results from Bronto clients.

Published in: Self Improvement, Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Create ContentIdentify 2 or more randomly selected groups – think about the size – make the groups large enough for the test results to be relevant.Define the testing variable (subject line, different from name address, content, date/time, and schedule to the groupsReview the results – look at all the metrics – open rates, click rates, conversion rates, amount of revenue, unsubsUtilize the winning results in future campaignsTest and Retest – Flukes happen. You can’t try something one time and be certain of anything. If you can, run the test at least 3 times to determine if your findings are actually a trend.
  • One test variable – when you test more than one variable, results get fuzzy and determining which variable made the difference becomes more difficult to determine.Test segments that are large enough to make the test valid. If your list is only 300 people, how valid is a 5% test segment? As a rule of thumb, the smaller the list, the larger the test groups should be.Focus on the right data points to determine the winner. Subject lines are typically used to drive opens, however they may drive clicks by being more call-to-action based. If you are trying to boost revenue, be sure to look at the full picture (which subject line will get the highest opens, which content changes generate the most clicks, etc).Allowenough time to elapse before determining a winner. Be sure that subscribers have had enough time to see, open, and interact with the message, typically a few hours should be the minimum. Realize that a group may initially look like it is losing but pull ahead at then end.Test at the appropriate time of the day. Unless all of your audience is comprised of night owls, it would not make sense to test at 2 AM and then send a winning message at 7 AM.Choose the right kind of split test (in Bronto, the choice would be to allow Bronto to automatically identify the winner and then send to the remaining audience hours later OR to run the test, manually review the results, and decide which action to take). While the automated option is a great way to optimize single send messages, some testing, such as when testing a message to be used for trigged campaigns should be given plenty of time to run before evaluating the results.ALWAYS BE TESTING – make testing an integral part of your email program. If nothing else, execute automated subject line tests with your sends.Pay attention to the results and adjust accordingly.
  • Testing TypesChampion Challenger – When you want to evaluate the results yourself. A/B Winner – One time send, limited time, want to allow Bronto to determine the winner based on specified criteria and automatically send the winning message to a set of contactsSplit OptionsWinning Criteria– when automating the testing with the A/B winner, you can have Bronto determine the winner based on Open Rates, Click Rates, Conversion Rates, Total Revenue, Revenue per Contact, Revenue per clickComparison reporting – allows you to compare the results of the split side by side so you can easily identify how each rate was affected by the variable
  • Schedule A/B WinnerSubject LineEvaluate on Click Rate 10/10/80Subject 1: Free Shipping ‘til Tuesday on Bronto GearSubject 2: Bronto Gear – Click Me, Please!Send to Customer List5 hour test time
  • Transcript

    • 1. 5 Steps for A/B SplitTestingJune 28, 2012
    • 2. Bronto Webinars Thank you for attending Housekeeping Q&A Recording @bronto Topic Suggestions? Bronto Nation
    • 3. Our Contest
    • 4. 5 Steps for A/B SplitTestingJune 28, 2012
    • 5. Agenda 3 A/B Split Testing in Bronto 2 Best Practices 1 A/B Split Testing Overview
    • 6. OverviewWhat is A/B Split Testing • Method of experimenting with email campaigns to determine variables that generate the best results • Results typically measured by • Open Rates • CTR • Conversion Rates • Total Revenue
    • 7. OverviewWhat can A/B Split TestingDetermine? • What day of week gets better open rates? • Which subject type (personalized, generic, call to action, shorter, longer) gets the best open rates? • Is it better to use your company name or a personal name in the from address? • Does one layout generate more revenue than another? • Many others….
    • 8. Steps for A/B Testing5 Steps for A/B Testing 1. Create content 2. Identify 2 or more randomly selected groups of contacts 3. Send message to the groups with a variable • Subject line • From name and address • Different Content • Different Date/Time of send 4. Review results 5. Takeand Re-test! winner Test action with
    • 9. Best Practices 1. Stick to one test variable. 2. Utilize segments large enough to make the test valid. 3. Focus on the right data points. 4. Allow enough time to elapse. 5. Test at the appropriate time. 6. Choose the right kind of split test (automated / manual). 7. Always Be Testing. 8. Pay attention to the results.
    • 10. A/B Split Testing in Bronto  Two types of testing • Champion / Challenger • A/B Winner  4 Split Options • Subject Line • From Name / Address • Date / Time of Send • Full Message Content  Multiple winning criteria • Open Rate • Click Rate • Revenue • Etc.  Comparison Reporting
    • 11. A/B Split TestingDemonstration
    • 12. Client Successes with A/B TestingGourmet Food Retailer – Send Time Optimization Test 549% Increase in Campaign RevenueClothing Retailer – Subject Line Testing Click Rate Increase of 23%Furniture Maker – Email Header Test Click Rate Increase of 37%
    • 13. Which Test Won
    • 14. Thanks for Joining!
    • 15. Thank You for Attending! Account Management Follow Up Email Webinar Recording Blog Post Which Test Won
    • 16. Contest Random Name Drawing Medium Inflatable Bronto!!!