SlideShare a Scribd company logo
1 of 31
Download to read offline
By Michaela Simpson
Student MSc (Management) Digital Marketing
DCU, 2013/14
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
E-mail budgets are increasing as Return on
Investment explodes to 2,500% (DMA, 2014)
Introduction to A/B Testing in E-mail Marketing
DMA, 2014
Introduction to A/B Testing in E-mail Marketing
www.communigator.co.uk
Google Dan Zarrella
http://blog.kissmetrics.com/science-of-social-timing-2/
http://blog.kissmetrics.com/science-of-social-timing-2/
1.1.HypothesisHypothesis
2.2.One VariableOne Variable
3.3.Clear Success MetricClear Success Metric
4.4.Volume and Statistical SignificanceVolume and Statistical Significance
5.5.RandomisationRandomisation
6.6.Always be Testing but Apply Common SenseAlways be Testing but Apply Common Sense
7.7.DocumentationDocumentation
(Search Engine Watch, 2012)(Search Engine Watch, 2012)
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Sent a test e-mail to yourself and to other
colleagues
Checked these on different browsers and
devices, including mobile devices
Checked all links within the e-mail are working
and relevant
Checked spelling and grammar
And when you have carefully
checked all of that….
Introduction to A/B Testing in E-mail Marketing
Introduction to A/B Testing in E-mail Marketing
Compare your open rate, click rate and bounce
rate with the industry.
http://mailchimp.com/resources/research/email-marketing-benchmarks/
Compare your open rate and click rate with
your own average.
Be aware of how many of your customers are
reading their e-mails on their mobile device.
E-mail is not dead!
When exploring A/B testing, explore Dan
Zarrella’s work.
Before setting up your A/B test follow the rules
Only send your e-mail when you’ve checked and
re-checked your campaign. Be anal!
Measure against industry averages and your own
averages.
@MichaelaTweets@MichaelaTweets
/michaelasimpson1/michaelasimpson1
www.michaela85.wordpress.comwww.michaela85.wordpress.com

More Related Content

Similar to Introduction to A/B Testing in E-mail Marketing

How to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your BusinessHow to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your BusinessKissmetrics on SlideShare
 
Questions On Teachers Background, Values And Beliefs
Questions On Teachers Background, Values And BeliefsQuestions On Teachers Background, Values And Beliefs
Questions On Teachers Background, Values And BeliefsRebecca Diamond
 
A/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data ScienceA/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data ScienceUncodemy
 
Building an Email Marketing Practice
Building an Email Marketing PracticeBuilding an Email Marketing Practice
Building an Email Marketing PracticeChris Sietsema
 
Ben Brown Resume
Ben Brown ResumeBen Brown Resume
Ben Brown ResumeBen Brown
 
Why email isn’t dead
Why email isn’t deadWhy email isn’t dead
Why email isn’t deadWill & Grail
 
Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Caitlin Wilson
 
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTION
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTIONDATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTION
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTIONCSEIJJournal
 
Marketing Communication Campaign Evaluation
Marketing Communication Campaign EvaluationMarketing Communication Campaign Evaluation
Marketing Communication Campaign EvaluationSatyam Arora
 
Email Marketing - From Best Practices to Best Results - Gina Nykerk
Email Marketing - From Best Practices to Best Results - Gina NykerkEmail Marketing - From Best Practices to Best Results - Gina Nykerk
Email Marketing - From Best Practices to Best Results - Gina NykerkBrandAidConference
 
Measuring the Results: How to Measure the Results of Email Communications and...
Measuring the Results: How to Measure the Results of Email Communications and...Measuring the Results: How to Measure the Results of Email Communications and...
Measuring the Results: How to Measure the Results of Email Communications and...Hobsons
 
NOVA Case Study Essay
NOVA Case Study EssayNOVA Case Study Essay
NOVA Case Study EssayBeth Hall
 
Problems With Manual Traceability Data For Large Software...
Problems With Manual Traceability Data For Large Software...Problems With Manual Traceability Data For Large Software...
Problems With Manual Traceability Data For Large Software...Melissa Wilson
 

Similar to Introduction to A/B Testing in E-mail Marketing (20)

Saucier_Jonathon_Resume
Saucier_Jonathon_ResumeSaucier_Jonathon_Resume
Saucier_Jonathon_Resume
 
Using Structured And Unstructured Data Essay
Using Structured And Unstructured Data EssayUsing Structured And Unstructured Data Essay
Using Structured And Unstructured Data Essay
 
How to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your BusinessHow to Use Data to Inform Your Design and Drive Your Business
How to Use Data to Inform Your Design and Drive Your Business
 
Questions On Teachers Background, Values And Beliefs
Questions On Teachers Background, Values And BeliefsQuestions On Teachers Background, Values And Beliefs
Questions On Teachers Background, Values And Beliefs
 
A/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data ScienceA/B Testing and Experimentation in Data Science
A/B Testing and Experimentation in Data Science
 
Career resume
Career resumeCareer resume
Career resume
 
Building an Email Marketing Practice
Building an Email Marketing PracticeBuilding an Email Marketing Practice
Building an Email Marketing Practice
 
Ben Brown Resume
Ben Brown ResumeBen Brown Resume
Ben Brown Resume
 
Email Report
Email ReportEmail Report
Email Report
 
Why email isn’t dead
Why email isn’t deadWhy email isn’t dead
Why email isn’t dead
 
Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...Assessment System And Model Is Based On The Following...
Assessment System And Model Is Based On The Following...
 
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTION
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTIONDATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTION
DATA MINING FOR STUDENTS’ EMPLOYABILITY PREDICTION
 
Resume
ResumeResume
Resume
 
Marketing Communication Campaign Evaluation
Marketing Communication Campaign EvaluationMarketing Communication Campaign Evaluation
Marketing Communication Campaign Evaluation
 
Email Marketing - From Best Practices to Best Results - Gina Nykerk
Email Marketing - From Best Practices to Best Results - Gina NykerkEmail Marketing - From Best Practices to Best Results - Gina Nykerk
Email Marketing - From Best Practices to Best Results - Gina Nykerk
 
UKW CV FINAL
UKW CV FINALUKW CV FINAL
UKW CV FINAL
 
UKW CV FINAL
UKW CV FINALUKW CV FINAL
UKW CV FINAL
 
Measuring the Results: How to Measure the Results of Email Communications and...
Measuring the Results: How to Measure the Results of Email Communications and...Measuring the Results: How to Measure the Results of Email Communications and...
Measuring the Results: How to Measure the Results of Email Communications and...
 
NOVA Case Study Essay
NOVA Case Study EssayNOVA Case Study Essay
NOVA Case Study Essay
 
Problems With Manual Traceability Data For Large Software...
Problems With Manual Traceability Data For Large Software...Problems With Manual Traceability Data For Large Software...
Problems With Manual Traceability Data For Large Software...
 

Recently uploaded

The Strategic Ensemble Behind Effective Content
The Strategic Ensemble Behind Effective ContentThe Strategic Ensemble Behind Effective Content
The Strategic Ensemble Behind Effective ContentClearVoice
 
Nutrient Market India- market research.pptx
Nutrient Market India- market research.pptxNutrient Market India- market research.pptx
Nutrient Market India- market research.pptxShekharParentage
 
Online Press Release Distribution PR Wires Experts.pptx
Online Press Release Distribution PR Wires Experts.pptxOnline Press Release Distribution PR Wires Experts.pptx
Online Press Release Distribution PR Wires Experts.pptxPR Wires
 
Maintaining Brand Resilience in The Midst Of Competition.pdf
Maintaining Brand Resilience in The Midst Of Competition.pdfMaintaining Brand Resilience in The Midst Of Competition.pdf
Maintaining Brand Resilience in The Midst Of Competition.pdfAndri H.
 
Mint IPO Offerings by HT Media - Deck.pptx
Mint IPO Offerings by HT Media - Deck.pptxMint IPO Offerings by HT Media - Deck.pptx
Mint IPO Offerings by HT Media - Deck.pptxb2bmarketinghindusta
 
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch Deck
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch DeckPodvertise.fm - Podcast Advertising Marketplace - Startup Pitch Deck
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch DeckNedko Nedkov
 
A navigation of two creative processes Study
A navigation of two creative processes StudyA navigation of two creative processes Study
A navigation of two creative processes Studystuwilson.co.uk
 
Báo cáo Xu hướng Tiktok 2024 từ TIKTOK
Báo cáo Xu hướng Tiktok 2024 từ TIKTOKBáo cáo Xu hướng Tiktok 2024 từ TIKTOK
Báo cáo Xu hướng Tiktok 2024 từ TIKTOKA2Z Marketing
 
India_MarTech_and_CommerceTech_Report_2024.pdf
India_MarTech_and_CommerceTech_Report_2024.pdfIndia_MarTech_and_CommerceTech_Report_2024.pdf
India_MarTech_and_CommerceTech_Report_2024.pdfSocial Samosa
 
Content drivers for global brands: new innovative paths for your branded cont...
Content drivers for global brands: new innovative paths for your branded cont...Content drivers for global brands: new innovative paths for your branded cont...
Content drivers for global brands: new innovative paths for your branded cont...Vanksen
 
Socio-digital evolutions and micro-communities: what business opportunities d...
Socio-digital evolutions and micro-communities: what business opportunities d...Socio-digital evolutions and micro-communities: what business opportunities d...
Socio-digital evolutions and micro-communities: what business opportunities d...Vanksen
 
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...saivigneshvicky86
 
Holistic Website Health & Technical Tips for SEO Success
Holistic Website Health & Technical Tips for SEO SuccessHolistic Website Health & Technical Tips for SEO Success
Holistic Website Health & Technical Tips for SEO SuccessSearch Engine Journal
 
Marketing Team of 1, A Framework To Win!
Marketing Team of 1, A Framework To Win!Marketing Team of 1, A Framework To Win!
Marketing Team of 1, A Framework To Win!Joseph Skibbie
 
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)Aldridge Christian Seubelan
 
How to Get Off Spamhaus Blocklist - 5 Step Process
How to Get Off Spamhaus Blocklist - 5 Step ProcessHow to Get Off Spamhaus Blocklist - 5 Step Process
How to Get Off Spamhaus Blocklist - 5 Step ProcessQuickEmailVerification
 
Message-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanMessage-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanVWO
 
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...aravindharajp
 
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdf
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdfTAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdf
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdfSocial Samosa
 

Recently uploaded (20)

Marketing Strategy of Maruti Suzuki.docx
Marketing Strategy of Maruti Suzuki.docxMarketing Strategy of Maruti Suzuki.docx
Marketing Strategy of Maruti Suzuki.docx
 
The Strategic Ensemble Behind Effective Content
The Strategic Ensemble Behind Effective ContentThe Strategic Ensemble Behind Effective Content
The Strategic Ensemble Behind Effective Content
 
Nutrient Market India- market research.pptx
Nutrient Market India- market research.pptxNutrient Market India- market research.pptx
Nutrient Market India- market research.pptx
 
Online Press Release Distribution PR Wires Experts.pptx
Online Press Release Distribution PR Wires Experts.pptxOnline Press Release Distribution PR Wires Experts.pptx
Online Press Release Distribution PR Wires Experts.pptx
 
Maintaining Brand Resilience in The Midst Of Competition.pdf
Maintaining Brand Resilience in The Midst Of Competition.pdfMaintaining Brand Resilience in The Midst Of Competition.pdf
Maintaining Brand Resilience in The Midst Of Competition.pdf
 
Mint IPO Offerings by HT Media - Deck.pptx
Mint IPO Offerings by HT Media - Deck.pptxMint IPO Offerings by HT Media - Deck.pptx
Mint IPO Offerings by HT Media - Deck.pptx
 
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch Deck
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch DeckPodvertise.fm - Podcast Advertising Marketplace - Startup Pitch Deck
Podvertise.fm - Podcast Advertising Marketplace - Startup Pitch Deck
 
A navigation of two creative processes Study
A navigation of two creative processes StudyA navigation of two creative processes Study
A navigation of two creative processes Study
 
Báo cáo Xu hướng Tiktok 2024 từ TIKTOK
Báo cáo Xu hướng Tiktok 2024 từ TIKTOKBáo cáo Xu hướng Tiktok 2024 từ TIKTOK
Báo cáo Xu hướng Tiktok 2024 từ TIKTOK
 
India_MarTech_and_CommerceTech_Report_2024.pdf
India_MarTech_and_CommerceTech_Report_2024.pdfIndia_MarTech_and_CommerceTech_Report_2024.pdf
India_MarTech_and_CommerceTech_Report_2024.pdf
 
Content drivers for global brands: new innovative paths for your branded cont...
Content drivers for global brands: new innovative paths for your branded cont...Content drivers for global brands: new innovative paths for your branded cont...
Content drivers for global brands: new innovative paths for your branded cont...
 
Socio-digital evolutions and micro-communities: what business opportunities d...
Socio-digital evolutions and micro-communities: what business opportunities d...Socio-digital evolutions and micro-communities: what business opportunities d...
Socio-digital evolutions and micro-communities: what business opportunities d...
 
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...
IMPACT OF GOOGLE'S CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXP...
 
Holistic Website Health & Technical Tips for SEO Success
Holistic Website Health & Technical Tips for SEO SuccessHolistic Website Health & Technical Tips for SEO Success
Holistic Website Health & Technical Tips for SEO Success
 
Marketing Team of 1, A Framework To Win!
Marketing Team of 1, A Framework To Win!Marketing Team of 1, A Framework To Win!
Marketing Team of 1, A Framework To Win!
 
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)
2024 Brand Trends (Strategy for Branding, Marketing, Digital, Communication)
 
How to Get Off Spamhaus Blocklist - 5 Step Process
How to Get Off Spamhaus Blocklist - 5 Step ProcessHow to Get Off Spamhaus Blocklist - 5 Step Process
How to Get Off Spamhaus Blocklist - 5 Step Process
 
Message-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tidemanMessage-Market Fit VWO Workshop by Daphne tideman
Message-Market Fit VWO Workshop by Daphne tideman
 
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...
IMPACT OF GOOGLE CORE WEB VITALS UPDATE ON WEBSITE PERFORMANCE AND USER EXPER...
 
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdf
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdfTAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdf
TAM AdEx-A Pixelated view into Digital Advertising Trends for Y 2023.pdf
 

Introduction to A/B Testing in E-mail Marketing

Editor's Notes

  1. Hi everyone! I’m Michaela and I’m a current Masters Student in Digital Marketing in Dublin City University. Over the year I’ve learned quite a bit about e-mail marketing, but of course, when you come to actually doing it, it still feels a bit daunting. So I’ve put together this webinar to introduce the basics A/B testing in E-mail Marketing. I’ll walk through, step by step, exactly how you run an A/B test using Mailchimp.
  2. But, first things first…
  3. Well, contrary to popular belief, e-mail is far from dead. There is plenty of evidence that e-mail is one of the most successful and cost effective channels a Digital Marketer can use, however, in many companies, e-mail is not seen as old fashioned and just not trendy in digital marketing today.
  4. But the one thing bosses can’t argue with is being presented with ROI. This is a from research carried out by the Direct Marketing Association in the UK in 2013. They found that E-mail ROI has hit an unbelievable 2,500%. So it’s not surprising that respondents indicated that they plan to invest further in E-mail marketing.
  5. The ROI from e-mail marketing is also increasing. In 2012, it was found to return £21 pounds for every £1 pound spent and that was up to £25 pounds for every £1 pound spend in 2013. It’s very important to note though, that e-mail is not magic. It needs care and attention in order to get this kind of success out of it. As the DMA say here, 77% of ROI comes from segmented, targeted and triggered campaigns. Customers who have signed up to your newsletter, don’t really care about what you have to say unless it is of some sort of value to them. Segmenting your e-mail lists is key to ensuring that your customers are receiving only email content which is of relevance to them.
  6. One final interesting point from the DMA research. They found the best performing emails in 2013 to be the regular old newsletter, followed by welcome messages and customer surveys. I think that’s an interesting point to keep in mind. Particularly the customer surveys. If you have customers who are willing to answer a survey for you, not only do you gain really valuable feedback, but you also gain great material to possibly form the basis of a really insightful blog post or white paper that can feed into your content marketing.
  7. Getting started with A/B testing. So, if you’re not quite sure what an A/B test is, I would describe it simply as creating 2 random groups. Group 1 is a “Control” group and group 2 is a “Treatment” group. Your treatment group is going to receive some sort of special treatment and then you will see if it has had any effect, based on the difference in outcomes between them and your control group.
  8. Getting started with A/B testing. So, if you’re not quite sure what an A/B test is, I would describe it simply as creating 2 random groups. Group A is a “Control” group and group B is a “Treatment” group. Your treatment group is going to receive some sort of special treatment, like maybe you want to try out phrasing your subject line as a question and seeing if that has any impact on the open rate. If you keep everything else in the e-mail the same, the time you send it, the content of the e-mail and so on, you will be quite confident that if there is any outrageous variation between the two groups, that it is very likely down to the thing you were testing, in this case the subject line. So once you have a handle on A/B testing, in my opinion, the next thing you should do is
  9. Google Dan Zarrella. He is what they call, a social media scientist. He takes different things like Twitter, Facebook and E-mail marketing and methodically tests the hell out of them. He asks simple questions like, does saying “please retweet” increase the likelihood of being retweeted? By the way , it turns out it does!
  10. For instance, this is from Zarrella’s Science of E-mail marketing where he tests the open rates and click rates by day. He had access to a large collection of e-mail data from Hubspot and found that open rates and click rates on weekends show an increase compared to midweek. This doesn’t mean that this would be the same in every case, but it might be something which you’d want to carry out an A/B test on.
  11. He also tested to see if the time of day made any difference to e-mail open rates and click rates. He got this this indication that sending e-mails early in the morning seemed to have the most favourable open rates and click rates.
  12. I’ll quickly outline the rules of A/B testing before walking through, step by step, how to carry out your own A/B test using MailChimp. Firstly, you need a hypothesis where you should write down a short sentence that summarizes what you’re trying to prove. Secondly, and as I mentioned a few slides back, an A/B test can only be successful if you are testing only one variable while everything else stays constant. Third, you need to choose which metric is going to show which group wins. I’m going to mention all of these again when we’re doing the walkthrough, so don’t worry if you’re not really sure what this right now. Fourth, make sure that the sample that you are using for the test is large enough to be statistically significant. Fifth, make sure the groups are selected randomly. When you are using MailChimp – it looks after this random splitting for you. Like Zarrella, always be testing, but try to keep your head screwed on and think carefully about what is worth testing. Finally, once you’ve put the work in to carrying out these tests, make sure to document it so that the findings aren’t wasted. Again, Search Engine Watch, makes a good suggestion saying why not make a blog post out of your findings – this way you document your work and you feed your content marketing channel!
  13. Ok, so now I’m going to walk through, step by step, how to run an A/B test on MailChimp. They actually make it really simple so you shouldn’t have any trouble with it, but if you haven’t tried it before, this should help you if you get stuck anywhere.
  14. So when you log into your Mailchimp, you’ll arrive at your dashboard as usual. You just go up to Create Campaign in the top right.
  15. That’ll bring you to a screen like this, where you have a choice to select a regular campaign, or down here you have you’re A/B Split Campaign. Click select.
  16. If you’re Mailchimp account is very new, you may not see this screen. They don’t seem to allow A/B tests until there is some data in the account from previous campaigns, probably showing them that you have made yourself familiar with the basic features of mailchimp before delving into the more complicated stuff. So on this page, you select what you would like to test. So in our case we’re going to test two different subject lines. What this means down here is that the test is going to be sent to 20% of your total list initially before picking the winner and sending on to the remaining 80% of your list. So in my case, my list had about 2,500 emails in it so 250 would have received email A and 250 would have received email B. Like I mentioned previously, Mailchimp picks these test segments randomly for you. Finally on this page, you choose how long we want Mailchimp to wait before making its decision on a winner and we’ll select one day here. Click next
  17. Next you are prompted to choose your recipients. All of your lists will be listed here and you pick the one you want to send this campaign to.
  18. Next you set up your campaign info. We said we were going to test two different subject lines and see if that has any impact on open rates. Here, I’ve taken inspiration from one of Zarrella’s findings where he found that contrary to what was often thought, posing a question in an e-mail subject line, actually correlated with lower open rates. But this may not be the case for everyone so we’ll test it out and see how our list responds to a question in the subject line.
  19. The next steps are where you will spend most of your time which is designing your e-mail content.
  20. And once that’s completed, you’ll be ready to send your e-mail.
  21. But wait, if I’ve learned anything the hard way it’s this…
  22. When the test is complete, Mailchimp presents the results like this. You can see all of the different metrics and how each performed. Because we were testing subject lines, the best metric to determine the winner is open rate. It’s the best indicator of how a subject line has performed as it is the most prominent feature a user sees before deciding whether or not to open the mail. In this case, for another test I ran, Group B was the winner with 40% opening the mail versus 31%.
  23. Other ways to guage the performance of your campaigns is to compare your metrics to the industry averages. Mailchimp has a great resource of industry averages for email marketing campaigns where you can see how your campaigns are measureing up.
  24. You should also check how each campaign measures up to your own average metrics for that list. You’ll be able to see whether this campaign did better than usual and perhaps get some hints as to why that was. All of these insights can be used to inform your decisions for your future campaigns
  25. Finally, you should make sure not to neglect mobile users. If you check your stats for how many of your customers are reading their emails on a mobile device, you might be surprised. If this is a large proportion, you should make sure that all of your campaigns are optimised for mobile and read well on the smaller screen.
  26. So just a quick recap of the key takeaways of this webinar. E-mail is not dead! For digital marketers who are giving it the time, attention and investment it deserves, it is returning hugely impressive ROI. Careful segmentation of your customers is the key to seeing the kind of results we saw earlier. When exploring A/B testing explore Dan Zarrella’s work. In my opinion, it’s a great place to start as he presents this kind of thing in a nice simple way for beginners, and helps experienced e-mail marketers see the wood for the trees. Before setting up your a/b test, take note of the rules I went through and allow these to guide you in forming a hypothesis and deciding how to choose your winner. Only send your e-mail when you’ve checked and re-checked your campaign. If ever you are going to be anal, be anal before you send out an e-mail campaign! Measure your results against industry averages and your own averages to get more insight into how your campaign has performed.
  27. So thanks a million for watching. If you have any questions or want to connect, you can get my on my Twitter @MichaelaTweets, my LinkedIn michaelasimpson1 or you can follow my digital marketing blog at michaela85.wordpress.com. Thanks everybody!