By the end of this year, MarketingExperiments will have run approximately 260 controlled experiments. That equates to around 50,000 total hours of research. With the sheer volume of work done this year, it’s almost impossible for marketers in the field to keep up. In our next Web clinic, we’ll walk through the most important discoveries of the year so you can learn the progress MECLABS analysts and scientists have made. You’ll discover actual research-grounded answers to the following questions: • When is the best time to send an email? • How should I design my website navigation? • Can a brand really make an impact on conversion? • Is my page missing critical copy? • And more … You’ll also receive access to the broadest array of researchers from our lab so far this year. They’ll be talking through the experiments and answering your questions live from the MarketingExperiments broadcast studio.

1. The Top 5 Marketing Discoveries in 2013
The last 50,000+ hours of research distilled into 60 minutes

2. We’re sharing on Twitter!
#WebClinic

3. Today’s speakers
Austin McCraw
Senior Editorial
Analyst
MECLABS
Pamela Markey
Senior Director
Marketing
MECLABS
Jon Powell
Senior Manager
Research and
Strategy
MECLABS
Adam Lapp
Director
Services
Operations
MECLABS
Ben Filip
Senior Manager
Data Sciences
MECLABS

4. Research sampling published in 2013
275%
Conversion
48%
Conversion
109%
Conversion
155%
Conversion
188%
Conversion
68%
Revenue/Visit
ec < op < ct < lp ©
220%
Conversion
C = 4m + 3v + 2(i-f) - 2a ©
17%
Clickthrough
36%
Conversion
101%
Conversion
eme = rv(of+ i) – (f + a) ©
87%
Conversion
97%
Conversion
fSC = lT + dT ©
Opr > Oprn > Ocnn ©
43%
Conversion

5. Today, we are going to distill all of this research into a 60minute walkthrough of the top five discoveries in 2013.

6. Discovery #1:
There are Five Consistent Factors that
Impact Customer Response to Email Timing

7. Experiment: Background
Experiment ID: TP2087
Record Location: MarketingExperiments Research Library
Research Partner: Protected
Research Notes:
Background: A large financial institution offering a financial service requiring an application to
consumers.
Goal: To increase the amount of completed applications.
Primary Research Question: Of the send times tested, which time will result in the highest rate of
completed applications to delivered emails?
Approach: A/B multifactor sequential test

8. Experiment: Design
14 Total Email Sends During Week of Test
Monday
Tuesday
AM Email
1
AM Email
3
PM Email
2
Wednesday
AM Email
5
PM Email
4
Thursday
AM Email
PM Email
8
AM Email
9
7
PM Email
6
Friday
Saturday
AM Email
11
PM Email
10
Sunday
AM Email
13
PM Email
12
PM Email
14
The test had a total 14 treatment paths – each path had the same subject line and email.
The treatment values included sending two emails each day of the week, Monday through
Sunday, at 3 a.m. and 3 p.m.

9. ?
Which Time of Day Will Perform Best?
3 a.m. or 3 p.m.?

10. Results
13.5% increase in clickthrough
Sending at 3 p.m. increased clickthrough rate by at least 13.5%.
Clickthrough Rate
3:00 a.m.
3:00 p.m.
Percent Relative Change
!
1.076%
1.220%
13.5%
What You Need to Understand: Recipients were 13.5% more likely to click in an
email sent at 3 p.m.

11. ?
Which Day of the Week Will Perform Best?
Mon., Tues., Wed., Thurs., Fri., Sat., Sun.

12. Experiment: Clickthrough rate by day
Clickthrough Rate By Day
Mon
Tues
Wed
Thurs
Fri
Sat
Sun

13. Experiment: Clickthrough rate by day
23.2% increase in clickthrough
Sending on Sunday increased clickthrough rate by 23.2% over Tuesday.
Treatments
(in order of performance)
Percent Relative
Change
LOC
Tuesday
10.47%
-
-
Monday
10.90%
4.1%
72%
Thursday
11.01%
5.1%
82%
Friday
11.59%
10.7%
99%
Wednesday
11.64%
11.2%
99%
Saturday
11.84%
13.1%
99%
Sunday
!
Clickthrough
12.90%
23.2%
99%
What You Need to Understand: Recipients were 23.2% more likely to click in an
email sent on Sunday than one sent on Tuesday. Sunday also significantly outperformed every other day of the week.

14. Experiment: Interpretation
Sunday Reader
Possible Reasons for Higher Clickthrough on Sunday
• Less distractions on Sunday
• Advent of mobile attaches recipients to email 24/7
• Personal nature of financial product causes higher
clickthrough outside of work

15. Experiment: Interpretation
Weekday Reader
Possible Reasons for Lower Clickthrough During
Week:
• Higher level of distraction
• Customers are in “work” mode and less likely to be
interested in financial application

16. What we discovered
F
Key Principles
1.
There is no “one-size-fits-all” time or frequency for an email send.
2.
However, the universal goal of email timing is to synchronize your email’s delivery with the cognitive
psychology of the customer’s purchase cycle.
3.
There are five consistent factors from campaign to campaign that will impact your customers’ purchase cycle:
1.
2.
3.
4.
5.
Decision cycle – (How often a decision is required)
Utility – (How useful the email is perceived)
Relevance – (How relevant the email is externally and internally)
Nature of product – (How the product affects the customer emotionally)
Expectations – (How the email matches customer expectations)
View the Full Clinic Here:
MarketingExperiments.com/EmailTiming

17. Discovery #2:
Brand Does Not Make a Promise, It Creates
an Expectation

18. Experiment: Background
Experiment ID: TP1651
Record Location: MECLABS Research Library
Research Partner: The Boston Globe
Research Notes:
Background: One of the largest metropolitan print news sources in the United States.
Goal: To increase the number of home delivery subscriptions.
Research Question: Which offer page will result in the highest subscription rate?
Test Design: A/B variable cluster test

19. Experiment: Control
• The goal of the original
page was to get people into
the subscription process.
• The original page used a
template CMS structure
that did very little to
leverage the brand of The
Boston Globe.

20. Experiment: Control
Offer Page
Step 1
• The unbranded template is
used throughout the entire
conversion process.
Step 2
Step 3
Step 4

21. Experiment: Treatment
• The treatment slightly
adjusts the CMS template
to emphasize The Boston
Globe brand.

22. Experiment: Treatment
• Boston Globe branding is
made prominent throughout
the entire conversion
process.

23. Experiment: Side-by-side
Will emphasizing a well-known brand in the online
subscription process generate more response?

24. Experiment: Results
40% increase in total subscribers
The branded subscription path increased the rate of subscriptions
by 40.3%
Clickthrough Rate
Control (Non-Branded)
Treatment (Branded)
Percent Relative Change
!
1.32%
1.86%
40.3%
What You Need to Understand: By simply emphasizing the well-known brand of
The Boston Globe, the treatment subscription path generated 40.3% more
subscriptions than the generically branded control.

25. What we discovered
Key Definition
Brand is the aggregate experience of the value proposition

26. What we discovered
F
Key Principles
1. Brand represents the sum total of experiences in the market place, particularly those that connect to you.
2. Brand exists in the mind. It represents a form of mental shorthand. It stands for a decision collective
and represents the default choice. As such, it implies expectation.
3. Brand does not make a promise; it creates an expectation. The strength of the brand is derived not from
declaration, but through expectation.
View the Full Clinic Here:
MarketingExperiments.com/DoesBrandMatter

27. Discovery #3:
Usability Does Not Equal “Buyability”;
All Usability Claims Must Be Tested

28. Experiment: Background
Experiment ID: TP1283
Record Location: MECLABS Research Library
Research Partner: Protected
Research Notes:
Background: Italian e-commerce website offering cosmetics. The researchers were focusing on
testing different approaches to the “body” category page.
Goal: To increase the rate of conversion.
Primary Research Question: Which page will generate the highest rate of conversion?
Approach: A/B variable cluster test

29. Experiment: Control
Control – Category List
• The control listed all of the
main categories of “body”
products:
•
•
•
•
•
•
•
•
•
•
•
Deodorants
Moisturizers
Toner
Skin
Hair remover
Feet
Hands
Mouth
Scrubs and specials
Accessories
Combination offers
• Products are all listed below
by category as selected

30. Experiment: Control
Control – Category List
Is the category list at the top of
the page the most user-friendly
way to present the information?

31. Experiment: Treatment 1
T1 – Configurator
• Treatment 1 seeks to make
the page easier to use by
adding an interactive
configurator that enables
the visitor to customize the
products that show up
below.
• By Category
• By Objective
• By Product Line

32. Experiment: Treatment 2
T2 – Visual Categories
• Treatment 2 seeks to make
the page easier to use by
removing the category links
and simply featuring the
main categories with
images.

33. Experiment: Treatment 3
T3 – Navigation Links (text)
• Treatment 3 is a radical
approach that seeks to
make the process easier
by removing the “body”
category page
altogether, enabling the
visitor to choose their
category within the
navigation of the
homepage.

34. Experiment: Treatment 4
T4 – Navigation Links (visual)
• Treatment 4 is similar
to Treatment 3, only it
integrates a more visual
approach to the
categories within the
navigation.

35. Experiment: Test your intuition
Control – Category
List
T1 – Configurator
T3 – Navigation Links (text)
T2 – Visual Categories
T4 – Navigation Links (visual)

36. Results
20% increase in total conversions
Treatment 1 category page increased conversion rate by 20%.
Treatments
Clickthrough
Percent Relative Change
Control – Category List
1.04%
-
T1 – Configurator
1.25%
20%
T2 – Visual Categories
1.10%
6%
T3 – Navigation Links (text)
1.10%
5%
T4 – Navigation Links (visual)
1.10%
5%

37. Experiment: Results
Why did the configurator beat all of the other methods? Was it the usability?
If so, what made it more usable than the other methods?
Control – Category
List
T1 – Configurator

38. What we discovered
F
Key Principles
1. Focusing on usability can hinder an effective approach to optimization. It often undermines the micro-yes
approach and can ultimately hinder results.
2. An overemphasis on usability confuses the means with the end. The goal of our website is not usability,
but rather “buyability.”
3. Usability, when viewed properly, can be a valuable tool for helping marketers identify hidden psychological
costs in their conversion processes.
4. All usability claims must be tested.
View the Full Clinic Here:
MarketingExperiments.com/UsabilityMyth

39. Discovery #4:
Letter-Style Emails are More Effective than
Promotional Emails

40. Experiment: Background
Experiment ID: TP2137
Record Location: MECLABS Research Library
Research Partner: [Protected]
Research Notes:
Background: A large international media company focusing on increasing
subscription rates.
Goal: To increase the number of conversions based on the value proposition conveyed through
the email.
Primary Research Question: Which email will generate the highest conversion rate?
Approach: A/B multifactor split test

41. Experiment: Control
CONTROL: Promotional-style email
• Uses popular design principles to
create balance and hierarchy on the
page.
• Heavy use of images and graphics to
catch the reader’s attention.
• Multiple call-to-action buttons for
increased points of entry.
Subject Line: Open this now for Special Savings

42. Experiment: Treatment
TREATMENT: Letter-style email
• Designed to look and feel more
like a personal letter.
• Limited use of graphics and
images.
• One call-to-action button.
Subject Line: Get Unlimited Access to [Product] with
Home Delivery

43. Experiment: Side-by-side
Treatment
Control
Subject Line: Get Unlimited Access to [Product]
with Home Delivery
Subject Line: Open this now for Special Savings
VS.

44. Experiment: Results
181% increase in conversion
The overall conversion rate increased 181% due to a
clearly stated value proposition.
Conversion Rate
Treatment – Letter-Style Email
0.04%
0.12%
Relative Difference
181%
Control – Standard Email
!
What You Need to Understand: By limiting the amount of graphics, and focusing on engaging the
customer in a conversation, the treatment outperformed the control by 181%.

45. Why did the letter-style win?
Subject Line: Get Unlimited Access to [Product] with
Home Delivery
Why did the
letter-style email win?

46. What we discovered
F
Key Learnings
1. An email message is not a monologue; it is a dialog. People don’t buy from emails;
people buy from people.
2. If the marketer can learn to participate with the prospect’s conversation, they can
guide it (with messaging) toward a satisfactory conclusion (the purchase).
3. Therefore, effective email messaging requires one often overlooked skill on the part
of the marketer: empathy.
View the Full Clinic Here:
MarketingExperiments.com/LetterEmails

47. What we discovered
Empathy: For the marketer, empathy is their ability to discern – through
listening and hearing – the ontology (nature or being) of the customer.
• Selfishness, if a benign version, is the primary driver of sales velocity. It is the selfishness of the
prospect which empowers the transaction.
• The concept, selfishness, has a negative connotation. But this can be unfortunate – how can a
self be faulted for being self(ish)? In one sense, selfishness is essence(tial).
• Empathy enables the marketer to identify with the market and experience its “selfishness.”
Empathy is the marketer’s intuition.

48. Discovery #5:
Copy Should Establish the Problem before
Presenting the Solution

49. Experiment: Background
Experiment ID: TP1700
Record Location: MECLABS Research Library
Research Partner: [Protected]
Research Notes:
Background: A medium-sized company selling a single auto repair product.
Goal: To increase the number of purchases.
Primary Research Question: Which landing page will generate the highest conversion rate?
Approach: A/B multifactor split test

50. Experiment: Control
Logo
Brand

51. Experiment: Treatment
Logo
Brand

52. Experiment: Side-by-side
Control
Treatment

53. Experiment: Results
36% increase in purchases
Product purchases increased by 36% over the control.
Conversion Rate
Control
Treatment
Relative Difference
!
1.33%
1.81%
36%
What You Need to Understand: By clarifying the problem before presenting the solution, the
treatment generated a 36% higher purchase rate than the control.

54. What we discovered
F
Key Principles
1.
Simply identifying a customer need does not inspire the need to act. The problem must be
intensified so that it is properly felt by customers.
2.
To inspire action, your copy must sufficiently transform a customer need into a customer want. This
can be achieved by using one or more of three problem intensifiers:
•
•
•
Relevance: the degree to which an offer is connected to a recipient’s situational motivations.
Importance: the degree to which an offer is essential to a recipient’s livelihood.
Urgency: the degree of immediacy associated with an offer.
View the Full Clinic Here:
MarketingExperiments.com/MissingCopy

55. Upcoming Research in 2014

56. EARLY BIRD SAVINGS

57. Experiment: Background
Experiment ID: TP1933
Record Location: MECLABS Research Library
Research Partner: [Protected]
Research Notes:
Background: A large news media organization trying to determine whether they should invest in
responsive mobile design
Goal: To increase the number of free trial sign-ups.
Primary Research Question: Which design will generate the highest rate of free trial sign-ups:
responsive or unresponsive?
Approach: A/B multifactor split test

58. Experiment: Control (Unresponsive)
Desktop
Mobile

59. Experiment: Treatment (Responsive)
Desktop
iPhone

60. Experiment: Side-by-side
Unresponsive
Responsive

61. Coming up in January 2014
Does Responsive Mobile Design
Actually Convert Better?

62. Summary: Putting it all together
Discovery #1:
There are Five Consistent Factors that Impact Customer Response to Email Timing
Discovery #2:
Brand Does Not Make a Promise, It Creates an Expectation
Discovery #3:
Usability Does Not Equal “Buyability”; All Usability Claims Must Be Tested
Discovery #4:
Letter-Style Emails are More Effective than Promotional Emails
Discovery #5:
Copy Should Establish the Problem before Presenting the Solution