At MarketingExperiments, 2015 has been another great year full of new research and discoveries about what makes customers say 'yes.' It’s not easy to attend or watch every Web clinic, so we want to provide you with the opportunity to catch up on what you may have missed.
In this 60-minute webinar, we looked back at our thousands of hours of research from 2015 to bring you five of the most surprising, actionable discoveries from the year. These simple principles will help you to quickly improve multiple aspects of your digital campaigns — from subject lines and calls-to-action to visitor engagement and mobile design.
With 2016 fast approaching, now is the time to make sure that all of the elements of your digital marketing campaigns are as effective as they can be. Join us on December 9 to learn quick, actionable tactics for improving your marketing collateral in time for the new year.
6. Experiment: Background
Background: A company offering a variety of dieting programs and memberships with the
goal of helping their audience lead a healthier lifestyle.
Goal: To increase landing page membership conversions on mobile and tablet devices.
Research Question: Which use of video will generate the highest conversion rate?
Test Design: Multi-treatment variable cluster split
Experiment ID: TP 11157
Record Location: MECLABS Research Library
Research Partner: (Protected)
8. Design Conversion Rate Level of Confidence % Relative Change
Control 0.14% ---
Treatment A 0.19% 99% 34.4%
Treatment B 0.17% 97% 22%
Treatment C 0.16% 89% 14.5%
Experiment: Mobile results
Relative increase in conversion rate34%Treatment A resulted in a 34.4% relative increase in conversion rate.
9. Two questions:
1. What common thing did the three treatments do (in different
ways) to change the customer experience?
11. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
Top 5 Marketing Discoveries in 2015
15. #WebClinic
Experiment: Background
Background: A tourism commission seeking to enhance visitor interaction with their website
content in order to boost appeal for choosing their city over other destinations.
Goal: Increase visitor engagement with site content.
Primary Research Question: Which treatment will increase site engagement?
Test Design: A/B/C split test
Experiment ID: TP 11211
Record Location: MECLABS Research Library
Research Partner: (Protected)
17. #WebClinic
Experiment: Treatments
*Anonymized
Treatment A
Treatment B
• Primary navigation tabs were
streamlined based upon visitor
interaction data with site content
over a calendar year.
• The most engaged-with category
content and secondary content were
prioritized and emphasized, while
less popular content was removed in
order to reduce friction.
18. #WebClinic
Design Clickthrough Rate Level of Confidence % Relative Change
Control 19.39% ---
Treatment A 26.26% 99% 35.39%
Treatment B 19.58% 55% 0.99%
Experiment: Results
Relative increase in navigation CTR35%Treatment A resulted in a 35.39% relative increase in navigation clickthrough rate.
19. #WebClinic
F
Design Lead Gen Rate Level of Confidence % Relative Change
Control 1.01% ---
Treatment A 1.36% 99% 34.93%
Treatment B 1.19% 91% 18.67%
Experiment: Results
Relative increase in total leads34%Treatment A resulted in a 34.93% relative increase in total leads.
What you need to understand:
Navigation content was curated based upon visitor behavior with the control nav.
20. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
Top 5 Marketing Discoveries in 2015
23. Experiment #1: Background
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in the area.
Research Question: Which subject line will generate the most opens and clickthrough?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.
Experiment ID: TP 20211
Record Location: MECLABS Research Library
Research Partner: (Protected)*
24. Experiment #1: Version A
SUBJECT: Are you a fall foodie? See what’s
going on in Boston this November.
25. Experiment #1: Version B
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun
26. Experiment #1: Which subject line will win?
Version A
SUBJECT: Are you a fall foodie? See what’s going on in Boston this
November
SUBJECT: Clam Chowder, Lobster Rolls & More: Calling All Foodies
for Boston’s Best Fall Fun
Version B
27. Experiment #1: Results
Relative increase in opens10%
Version
Open
Rate
Relative
Difference
Level of
Confidence
Version A 19.8% -- 99%
Version B 21.7% 10% 99%
28. Experiment #1: Results
Relative increase in clickthrough rate15%
Open
Rate
Relative
Difference
Level of
Confidence
Version A 2.7% -- 99%
Version B 3.1% 15% 99%
29. Experiment #2: Background
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in the area.
Research Question: Which subject line will generate the most opens and clickthrough?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.
Experiment ID: TP 20219
Record Location: MECLABS Research Library
Research Partner: (Protected)*
30. Experiment #2: Version A
SUBJECT: Holiday shopping, historic tours, Pops
Orchestra concerts and more Boston traditions
31. Experiment #2: Version B
SUBJECT: Book now to save up to 30% on hotel
stays this December in Boston
32. Experiment #2: Which subject line will win?
Version A
SUBJECT: Book now to save up to 30% on hotel stays this December in
Boston
SUBJECT: Holiday Shopping, Historic Tours, Pops Orchestra
Concerts and more Boston Traditions
Version B
33. Experiment #2: Results
Relative increase in opens9%
Version
Open
Rate
Relative
Difference
Level of
Confidence
Version A 19.3% 9% 99%
Version B 17.6% -- 99%
34. Experiment #2: Results
Relative increase in clickthrough rate26%
Version
Click
Rate
Relative
Difference
Level of
Confidence
Treatment 3.1% 26% 99%
Control 2.5% -- 99%
35. What do winning subject lines have in common?
SUBJECT: Are you a fall foodie? See what’s
going on in Boston this November
SUBJECT: Holiday shopping, historic tours,
Pops Orchestra concerts and more Boston
traditions
SUBJECT: Book now to save up to 30% on
hotel stays this December in Boston
SUBJECT: Clam Chowder, Lobster Rolls and
More: Calling all foodies for Boston’s best
fall fun
Losing Versions Winning Versions
Why?
36. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
Top 5 Marketing Discoveries in 2015
37. Experiment #1:
Loser
SUBJECT: Are you a fall foodie? See what’s going on in
Boston this November
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun
Winner
• Confusing
• Vague
38. Experiment #1:
Loser
SUBJECT: Are you a fall foodie? See what’s going on in
Boston this November
SUBJECT: Clam Chowder, Lobster Rolls and More:
Calling all foodies for Boston’s best fall fun
Winner 10% opens
Specific
Subject Line
• Tangible
• Clear
• Confusing
• Vague
40. Experiment #2:
Loser
SUBJECT: Book now to save up to 30% on hotel
stays this December in Boston
SUBJECT: Holiday shopping, historic tours, Pops
Orchestra concerts and more Boston traditions
Winner 9% opens
Specific
Subject Line
• Specific
• Valuable
• Salesy
• Ambiguous
42. Experiment: Background
Background: Think tank soliciting year-end donations to meet annual fundraising goal.
Goal: To increase donations.
Research Question: Which email will generate the most revenue?
Test Design: A/B split test
Experiment ID: #616
Record Location: NextAfter Digital Research Library
Research Partner: The Heritage Foundation
43. Experiment: Version A
• This email was sent by Jim DeMint,
the president and most well-known
leader of The Heritage Foundation,
asking recipients for their support
with a year-end gift
• The tone of the email is formal and
professional
44. Experiment: Version B
• The email was sent by Christie
Fogarty, the foundation’s lesser
known Director of Membership
Programs, requesting donations
• The email uses a much friendlier,
informal tone
47. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
4. The growing effectiveness of an “everyday tone” in email
People still don’t buy from (or give to) companies, stores or websites; people buy from people.
Top 5 Marketing Discoveries in 2015
48. An Everyday Tone
• The winning email hypothesized
(and confirmed) that a lesser-
known sender with a shorter,
briefer message would imply a
more believable, personal email
send
49. An Everyday Tone
• The winning email uses language
that more closely aligns to the
personal nature of email
• In fact, it was designed to read like
a conversation and is true to the
style of the actual sender
50. An Everyday Tone
Version B
• The winning email is plain about
the purpose of the ask and how it
will be used
• In addition, the call-to-action is
very clear and direct
53. Experiment: Background
Experiment ID: TP11081
Record Location: MECLABS Research Library
Research Partner: Protected*
Background: A regional marketing commission.
Goal: To raise awareness of local activities and events, increasing number of travelers and
tourist spending in [City].
Research Question: Which sign-up page will generate the most response?
Test Design: A/B split test
*Note: Test has been anonymized to protect partner confidentiality.
61. 5. The impact of brand standards on end-behavior
We must ensure that our defined elements of our brand help, rather than hinder, people when
it comes to making the right decision.
Top 5 Marketing Discoveries in 2015
64. 1. The impact of immediate authority in content marketing
Content is not free. There may not be a cost financially, but there is always a cost
psychologically.
2. The impact of intentional navigation in initial engagement
Having relevant content is not enough. We must learn to not only optimize the essence (focus)
of our content, but also it’s expression (form).
3. The impact of proper specificity in the email subject line
Many marketers worry about their ability to write persuasive copy, but the marketer’s art is
not persuasion; it is clarity.
4. The growing effectiveness of an “everyday tone” in email
People still don’t buy from (or give to) companies, stores or websites; people buy from people.
Review: Top 5 Marketing Discoveries in 2015
65. 5. The impact of brand standards on end-behavior
We must ensure that our defined elements of our brand help, rather than hinder, people when
it comes to making the right decision.
Review: Top 5 Marketing Discoveries in 2015