The Formula for Email
and Web Conversion
Success
Jon Powell
MECLABS
MarketingSherpa | MarketingExperiments
Top Obstacles in Email
Staffing, expertise
Merging Email Data

Creating consistent, relevant content
Evaluates
performance
from:
• 1 billion+ sent and
tracked email
messages
• Over 400 unique
rules

messages
rules

1 billio...
Anmessaging effectiveness
Experiment
Email
Test Scenario 1



Experiment ID: (protected)
Location: MarketingExperiments Research Library
Test Protocol Number: TP209...
Test Scenario 1
Test Protocol ID: TP2092
Version A

Looking for a lower mortgage rate?
You’re not alone.

SL Rating

63

V...
Accurate Prediction



Version B & C generated a statistically significant improvement over A

Subject Line

Tool
Rating
...


Accurate Prediction
Mostly Accurate Prediction
Version C still generated a statistically significant improvement over A...
Test Scenario 2



Experiment ID: (protected)
Location: MarketingExperiments Research Library
Test Protocol Number: TP207...
Test Scenario 2
Test Protocol ID: TP2078
Version A

Choose from 2 ways to save big on
the NEW [PublicationName].com

SL Ra...
Inaccurate Prediction
While Version A generated a statistically significant response, Version B did not
Subject Line

Tool...
Inaccurate Prediction
While Version A generated a statistically significant response, Version B did not
Subject Line

Tool...
Test Scenario 3
Test Protocol ID: TP2076
Version A

Target MDs with Surveys and Panels

SL Rating

94

Version B

Reach 12...
Inaccurate Prediction
Version B generated a statistically significant higher response than A

Subject Line

A Target MDs w...
Inaccurate Prediction
Version B generated a statistically significant higher response than A

Subject Line

A Target MDs w...
Experiment: Results
After rating hundreds of subject lines from our library of statistically
significant experiments…
The ...
The tool predicts a winner:

A coin predicts a winner:

31.57%

50%

of the time

of the time

25¢ Subject Line Rating Too...
So what went wrong with the tool?
Decisions based on violations:
•
•
•
•

•
•
•
•
•
•
•

Does not contain a sense of urgen...
So what went wrong with the tool?
Decisions based on violations:
•
•
•
•

Does not contain a sense of urgency
Mobile subje...
The computer approach
• Computer logic draws
conclusions from
algorithms, rules,
switches and settings
• Human logic draws...
The computer approach
In marketing…

We tend to rely on rule sets
•
•
•
•
•
•
•

Emphasize the benefits
Avoid long copy, u...
The Research
MarketingExperiments was the first Internet-based
research lab to conduct experiments in optimizing
the conve...
The Research
MarketingExperiments was the first Internet-based
research lab to conduct experiments in optimizing
the conve...
The Result
Email Messaging Optimization Heuristic

eme = rv(of + i) – (f + a)

©

eme = email messaging effectiveness inde...
The Result

eme = rv(of+i) – (f+a)
The Result
eme = rv(of + i) – (f + a)
Perceived Value

eme = rv(of+i) – (f+a)

©

Perceived Cost

a f

i

of
Relevance (rv)
Increasing Perceived Value
Relevance
eme = rv(of + i) – (f + a) ©
Relevance -- The compatibility of the email message to
the recipient’s motivations.
Relevance
From This

eme = rv(of+i) – (f+a)

To This
Relevance

F

Key Principles

Internal motivations
•
•
•
•
•
•

eme = rv(of+i) – (f+a)

Personal interests
Demographics
Sh...
Experiment



eme = rv(of+i) – (f+a)

Experiment ID: Pier 1 Imports
Location: MarketingSherpa Research Library
Test Proto...
Experiment

eme = rv(of+i) – (f+a)

Sample
Team A
Team B
Team C
Experiment
Team A

eme = rv(of+i) – (f+a)

Team B

Team C

Sample
18-52% Decrease in Click-through
Each design achieved a statistically significant click-rate decrease

Email Designs

Clic...
eme = rv(of+i) – (f+a)

Experiment
Click-through rate comparison

Team A

Team C

Team B

Client

17.68%

24.07%

29.91%

...
eme = rv(of+i) – (f+a)

Experiment
See a progression?

Team A

Team C

Team B

Client

17.68%

24.07%

29.91%

36.70%
Offer (of)
Increasing Perceived Value
Offer
eme = rv(of + i) – (f + a) ©
Offer -- The value you promise in exchange for an action
(i.e. a click, a transaction)
Offer

eme = rv(of+i) – (f+a)

What really is the offer in email?
• You may think that the offer is the product that you w...
Offer

eme = rv(of+i) – (f+a)
Experiment



eme = rv(of+i) – (f+a)

Experiment ID: (Protected)
Location: MarketingExperiments Research Library
Test Pro...
Experiment

eme = rv(of+i) – (f+a)

Control
PROTECTED

Get started with a free
30-minute demo

Get started with a free 30-...
Treatment
This version
focuses less on
platform and
more on its users
The primary callto-action was
written to
synchronize...
Experiment
Control

eme = rv(of+i) – (f+a)

Treatment
104% Increase in clicks
Version B achieved a statistically significant increase in click-rate

Email

Click Rate

Rel. Dif...
Offer

F

eme = rv(of+i) – (f+a)

Key Principles

• In the body of the email, it is important to distinguish
the differenc...
Offer

F

eme = rv(of+i) – (f+a)

Key Principles

• In the body of the email, it is important to distinguish
the differenc...
Offer

eme = rv(of+i) – (f+a)

Test Protocol ID: TP2076
Not this…

Target MDs with Surveys
and Panels
But this…

Reach 120...
Offer

eme = rv(of+i) – (f+a)

From This

To This
Incentive (i)
Increasing Perceived Value
Incentive
eme = rv(of + i) – (f + a) ©

Incentive – An additional element introduced to achieve a
desired action.
Incentive

F

eme = rv(of+i) – (f+a)

Key Principles

• The purpose of an incentive is to increase the probability
of imme...
Incentive

eme = rv(of+i) – (f+a)

We typically think of…
• Discounts (20% off, buy-one-get-one-free)

• Order/Delivery (F...
Incentive

eme = rv(of+i) – (f+a)

Also consider including…
• Instilled Deadlines

• Standard promotions

24 hours only: F...
Incentive

F

eme = rv(of+i) – (f+a)

Key Principles

• The purpose of an incentive is to increase the probability
of imme...
Friction (f)
Decreasing Perceived Cost
Friction
eme = rv(of + i) – (f + a) ©
Friction – Psychological resistance to a given element
anywhere in the email process
Friction

F

eme = rv(of+i) – (f+a)

Key Principles

• Two common elements that create friction in the
process are:
• Leng...
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: TP2089

1

Length

But This
Not this

21.2%
Increase in clicks
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: TP2081

1

Length

But This

Not this

41.4%
Increase in clicks

52.5%...
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: TP2026

2

Difficulty
But this (1 Option)

Not this (6 Options)

20%
I...
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: TP2026

2

Difficulty

Not this

But This

10%
Increase in clicks
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: A-TP1002

What else contributed to differences in performance?
Not Thi...
Friction

eme = rv(of+i) – (f+a)

Test Protocol ID: A-TP1002

What else contributed to differences in performance?
Not Thi...
Anxiety (a)
Decreasing Perceived Cost
Anxiety
eme = rv(of + i) – (f + a) ©
Anxiety – Psychological concern stimulated by a given
(or unaddressed) element in the...
Anxiety

eme = rv(of+i) – (f+a)

They’re just hard water stains…
Anxiety

F

eme = rv(of+i) – (f+a)

Key Principles

• Anxiety is just as lethal to conversion as friction.
• Try and mitig...
Experiment



eme = rv(of+i) – (f+a)

Experiment ID: (Protected)
Location: MarketingExperiments Research Library
Test Pro...
Experiment
Control
Subject Line: Your Free RegOnline Access

Dear [Name],
You’re just one step away from getting FREE acce...
Treatment

Subject Line: Your Free RegOnline Access
Hi [First],

I noticed that you started the process of getting free ac...
Experiment
Control

Direct Response Tone

eme = rv(of+i) – (f+a)

Treatment

Customer Service
Tone
349% Increase in lead inquiry rate
The service tone achieved a statistically significant increase in inquiries

Email

Inq...
Experiment



eme = rv(of+i) – (f+a)

Experiment ID: (Protected)
Location: MarketingExperiments Research Library
Test Pro...
Experiment
#
ID1:
ID2:
ID3:
ID4:
ID5:
ID6:
ID7:
ID8:
ID9:
ID10:
ID11:
ID12:
ID13:

eme = rv(of+i) – (f+a)

Subject Line Te...
Experiment
#
ID1:
ID2:
ID3:
ID4:
ID5:
ID6:
ID7:
ID8:
ID9:
ID10:
ID11:
ID12:
ID13:

eme = rv(of+i) – (f+a)

Subject Line Te...
Experiment
#
ID1:
ID2:
ID3:
ID4:
ID5:
ID6:
ID7:
ID8:
ID9:
ID10:
ID11:
ID12:
ID13:

eme = rv(of+i) – (f+a)

Subject Line Te...
Experiment
#
ID3:
ID1:
ID9:
ID5:
ID7:
ID12:
ID10:
ID6:
ID8:
ID11:
ID4:
ID13:
ID2:

eme = rv(of+i) – (f+a)

Subject Line Te...
Live Optimization
Email Messaging Effectiveness
Live Optimization



eme = rv(of+i) – (f+a)

Partner ID: The New York Times
Location: MarketingExperiments Research Libra...
Live Optimization

eme = rv(of+i) – (f+a)

The Subscription Product Offering

A subscription to the NY Times
• Home delive...
The List
THE CONTROL
Subject Line:

Open this now
for Special Savings
BEFORE

AFTER
62% Increase in Conversions
The treatment increased the subscription conversion rate by 62.1%

Email +
Subject Line

Open
...
Review
Email Messaging Effectiveness
Review
Pay attention to the change in the mind; not the
change in the content

eme = rv(of + i) – (f + a) ©
Perceived Valu...
Review

F

eme = rv(of+i) – (f+a)

Key Principles

•

There’s a story behind everything that’s bought. This occurs on two
...
February 17-20
Aria Las Vegas

FLINT MCGLAUGHLIN

DAN ARIELY

NOAH GOLDSTEIN

The longest running
research-based
email mar...
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
LCMC Session: Email and Web Conversion Success
Upcoming SlideShare
Loading in...5
×

LCMC Session: Email and Web Conversion Success

516

Published on

Published in: Marketing, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
516
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

LCMC Session: Email and Web Conversion Success

  1. 1. The Formula for Email and Web Conversion Success Jon Powell MECLABS MarketingSherpa | MarketingExperiments
  2. 2. Top Obstacles in Email Staffing, expertise Merging Email Data Creating consistent, relevant content
  3. 3. Evaluates performance from: • 1 billion+ sent and tracked email messages • Over 400 unique rules messages rules 1 billion+ email 400 unique
  4. 4. Anmessaging effectiveness Experiment Email
  5. 5. Test Scenario 1  Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2092 Background: Large financial institution offering mortgages to consumers. Goal: To determine which claim of value will generate the most response. Primary research question: Which subject line will produce the greatest open rate and click rate? Approach: A/B/C single-factorial split
  6. 6. Test Scenario 1 Test Protocol ID: TP2092 Version A Looking for a lower mortgage rate? You’re not alone. SL Rating 63 Version B Discover the advantages of [Brand Name] mortgages 78 Version C Own your home sooner with free biweekly payments 81
  7. 7. Accurate Prediction  Version B & C generated a statistically significant improvement over A Subject Line Tool Rating Open Rate Rel. Diff. A Looking for a lower mortgage rate? You’re not alone. 63/100 18.08% - B Discover the advantages of [Brand Name] mortgages 78/100 19.80% 9.5% C Own your home sooner with free bi-weekly payments 81/100 20.99% 16.0%
  8. 8.  Accurate Prediction Mostly Accurate Prediction Version C still generated a statistically significant improvement over A B&C statistically significant improvement over A Subject Line Tool Rating Open Rate Click Rel. Rate Diff. Rel. Diff. A Looking for a lower mortgage rate? You’re not alone. 63/100 18.08% 0.61% - - B Discover the advantages of [Brand Name] mortgages 78/100 19.80% 0.57% 9.5% -7.4% C Own your home sooner with free bi-weekly payments 81/100 20.99% 16.0% 0.68% 11.3%
  9. 9. Test Scenario 2  Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2078 Background: A well-known news publication Goal: To increase subscriptions from former subscribers who have been absent between one and two years Primary research question: Which subject line will produce the greatest open rate? Approach: A/B/C single-factorial split
  10. 10. Test Scenario 2 Test Protocol ID: TP2078 Version A Choose from 2 ways to save big on the NEW [PublicationName].com SL Rating 56 Version B Save 50% on your choice of two new subscription options 59 Version C Home Delivery comes with FREE access to [PublicationName].com 50
  11. 11. Inaccurate Prediction While Version A generated a statistically significant response, Version B did not Subject Line Tool Rating Open Rate Rel. Diff. A Choose from 2 ways to save big on the NEW [PublicationName].com 56/100 9.43% 50.2% B Save 50% on your choice of two new subscription options 59/100 7.01% 11.7% C Home Delivery comes with FREE access to [PublicationName].com 50/100 6.28% --
  12. 12. Inaccurate Prediction While Version A generated a statistically significant response, Version B did not Subject Line Tool Rating Open Rate Click Rel. Diff. Rate Rel Diff. A Choose from 2 ways to save big on the NEW [PublicationName].com 56/100 9.43% 1.48% 50.2% 86% B Save 50% on your choice of two new subscription options 59/100 7.01% 0.76% 11.7% -5.3% C Home Delivery comes with FREE access to [PublicationName].com 50/100 6.28% -0.80% --
  13. 13. Test Scenario 3 Test Protocol ID: TP2076 Version A Target MDs with Surveys and Panels SL Rating 94 Version B Reach 120,000 Physicians through Social Media 76
  14. 14. Inaccurate Prediction Version B generated a statistically significant higher response than A Subject Line A Target MDs with Surveys and Panels B Reach 120,000 Physicians through Social Media Tool Rating Open Rate Rel Diff. 94/100 21.30% -- 76/100 22.92% 7.6%
  15. 15. Inaccurate Prediction Version B generated a statistically significant higher response than A Subject Line A Target MDs with Surveys and Panels B Reach 120,000 Physicians through Social Media Tool Rating Open Rate Rel Diff. 94/100 21.30% -- 76/100 22.92% 7.6%
  16. 16. Experiment: Results After rating hundreds of subject lines from our library of statistically significant experiments… The tool predicts a winner: 31.57% of the time
  17. 17. The tool predicts a winner: A coin predicts a winner: 31.57% 50% of the time of the time 25¢ Subject Line Rating Tool Check Yours Now!
  18. 18. So what went wrong with the tool? Decisions based on violations: • • • • • • • • • • • Does not contain a sense of urgency Mobile subject line is over 35 characters Length is over 50 characters Contains any word that has more than 8 characters Includes one exclamation mark Use of a word in all caps Includes the word “You” or “Your” Contains comma Starts with Save Use of $ Contains % off
  19. 19. So what went wrong with the tool? Decisions based on violations: • • • • Does not contain a sense of urgency Mobile subject line is over 35 characters Length is over 50 characters Contains any word that has more than 8 characters Includes one exclamation mark Use of a word in all caps Includes the word “You” or “Your” Contains comma Starts with Save Use of $ Contains % off Computers are great with • computer logic; • • But not •so great with • • human logic •
  20. 20. The computer approach • Computer logic draws conclusions from algorithms, rules, switches and settings • Human logic draws conclusions from context, emotions, desires and belonging.
  21. 21. The computer approach In marketing… We tend to rely on rule sets • • • • • • • Emphasize the benefits Avoid long copy, use bullets Always include a call to action Keep it above the fold Emphasize hero shots Minimize compliance text Use tabs for additional info
  22. 22. The Research MarketingExperiments was the first Internet-based research lab to conduct experiments in optimizing the conversion rate of sales and marketing processes. 1992 Preliminary Research Begins 1997 Research Program Established 2001 First Research Report Published 2002 Testing of Research Partnership Model Begins 2003 Offer Response Optimization Theory Validated 2006 Patent Filings for Research Findings (10 Heuristics)
  23. 23. The Research MarketingExperiments was the first Internet-based research lab to conduct experiments in optimizing the conversion rate of sales and marketing processes. People judge emails, websites in the same way they judge people 1992 Preliminary Research Begins 1997 Research Program Established 2003 Offer Response Optimization Theory Validated 2006 Patent Filings for Research Findings (10 Heuristics) It’s not the change on the page; 2001 First Research Report Published 2002 Testing of Research Partnership Model Begins It’s the change in the mind
  24. 24. The Result Email Messaging Optimization Heuristic eme = rv(of + i) – (f + a) © eme = email messaging effectiveness index rv = relevance to the consumer of = offer to act upon i = incentive to take action f = friction elements of process a = anxiety elements of process
  25. 25. The Result eme = rv(of+i) – (f+a)
  26. 26. The Result eme = rv(of + i) – (f + a) Perceived Value eme = rv(of+i) – (f+a) © Perceived Cost a f i of
  27. 27. Relevance (rv) Increasing Perceived Value
  28. 28. Relevance eme = rv(of + i) – (f + a) © Relevance -- The compatibility of the email message to the recipient’s motivations.
  29. 29. Relevance From This eme = rv(of+i) – (f+a) To This
  30. 30. Relevance F Key Principles Internal motivations • • • • • • eme = rv(of+i) – (f+a) Personal interests Demographics Shopping habits Personality Communication styles Level of engagement External events • Environmental changes, realities • Holidays • News events • Competitive initiatives
  31. 31. Experiment  eme = rv(of+i) – (f+a) Experiment ID: Pier 1 Imports Location: MarketingSherpa Research Library Test Protocol Number: A-TP1002 Background: B2C home products company with a significant online and retail presence Goal: To increase the click-through rate from the email to the landing page Primary research question: Which email design will generate the most clicks? Approach: A/B/C/D split test (variable cluster)
  32. 32. Experiment eme = rv(of+i) – (f+a) Sample
  33. 33. Team A
  34. 34. Team B
  35. 35. Team C
  36. 36. Experiment Team A eme = rv(of+i) – (f+a) Team B Team C Sample
  37. 37. 18-52% Decrease in Click-through Each design achieved a statistically significant click-rate decrease Email Designs Click Rate Rel. Diff. Sample (client) 36.70% - Team A Design 17.68% -51.83% Team B Design 29.91% -18.50% Team C Design 24.07% -34.41%
  38. 38. eme = rv(of+i) – (f+a) Experiment Click-through rate comparison Team A Team C Team B Client 17.68% 24.07% 29.91% 36.70%
  39. 39. eme = rv(of+i) – (f+a) Experiment See a progression? Team A Team C Team B Client 17.68% 24.07% 29.91% 36.70%
  40. 40. Offer (of) Increasing Perceived Value
  41. 41. Offer eme = rv(of + i) – (f + a) © Offer -- The value you promise in exchange for an action (i.e. a click, a transaction)
  42. 42. Offer eme = rv(of+i) – (f+a) What really is the offer in email? • You may think that the offer is the product that you want to sell. • In reality it is the additional information they receive in exchange for a click. • What is the most effective way to get a click? Is it to give them an entire product pitch? Or just a part?
  43. 43. Offer eme = rv(of+i) – (f+a)
  44. 44. Experiment  eme = rv(of+i) – (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2801 Background: A physician-only social network that allows medical product companies to conduct first-hand research on potential purchasers. Goal: To increase the number of leads from a rented trade publication list. Primary research question: Which email design will generate the most opens, clicks? Approach: Multi-factorial sequential test on a trade publication list
  45. 45. Experiment eme = rv(of+i) – (f+a) Control PROTECTED Get started with a free 30-minute demo Get started with a free 30-minute demo This email assumes that readers understand the platform’s audience and thus focuses on platform uses It also assumes intent to purchase with multiple “Get Started” calls to action, one as early as the second sentence
  46. 46. Treatment This version focuses less on platform and more on its users The primary callto-action was written to synchronize with their state of interest PROTECTED
  47. 47. Experiment Control eme = rv(of+i) – (f+a) Treatment
  48. 48. 104% Increase in clicks Version B achieved a statistically significant increase in click-rate Email Click Rate Rel. Diff. Control 1.55% -- Treatment 3.16% 104%
  49. 49. Offer F eme = rv(of+i) – (f+a) Key Principles • In the body of the email, it is important to distinguish the difference between the product offer and the clickthrough offer.
  50. 50. Offer F eme = rv(of+i) – (f+a) Key Principles • In the body of the email, it is important to distinguish the difference between the product offer and the clickthrough offer. • Your communication of value starts with your subject line and should progress, like a conversation, throughout the entire conversion process
  51. 51. Offer eme = rv(of+i) – (f+a) Test Protocol ID: TP2076 Not this… Target MDs with Surveys and Panels But this… Reach 120,000 Physicians through Social Media 27.6% Increase in clicks
  52. 52. Offer eme = rv(of+i) – (f+a) From This To This
  53. 53. Incentive (i) Increasing Perceived Value
  54. 54. Incentive eme = rv(of + i) – (f + a) © Incentive – An additional element introduced to achieve a desired action.
  55. 55. Incentive F eme = rv(of+i) – (f+a) Key Principles • The purpose of an incentive is to increase the probability of immediate action by tipping the balance of emotional conflict inside the person
  56. 56. Incentive eme = rv(of+i) – (f+a) We typically think of… • Discounts (20% off, buy-one-get-one-free) • Order/Delivery (Free shipping, accelerated shipping, instant download • Add-ons (subscription extensions, warrantee extensions) • Content (Whitepapers, books, articles, reports , e-books) • Accompanying product Incentives • Free accessories (carrying case) • Complementary products (memory card)
  57. 57. Incentive eme = rv(of+i) – (f+a) Also consider including… • Instilled Deadlines • Standard promotions 24 hours only: Free shipping on… • Implied exclusivity 48-hour member exclusive: new fall styles… • Contests This is the last day to enter for your chance to… • Limited Availability Only 25 seats left to see register for...
  58. 58. Incentive F eme = rv(of+i) – (f+a) Key Principles • The purpose of an incentive is to increase the probability of immediate action by tipping the balance of emotional conflict inside the person • In email messaging, it can also include the use of artificial urgency, in which the immediacy associated with the offer is controlled by the marketer.
  59. 59. Friction (f) Decreasing Perceived Cost
  60. 60. Friction eme = rv(of + i) – (f + a) © Friction – Psychological resistance to a given element anywhere in the email process
  61. 61. Friction F eme = rv(of+i) – (f+a) Key Principles • Two common elements that create friction in the process are: • Length • Difficulty • The objective is to minimize, not eliminate friction. If you eliminate all friction, you eliminate “the sale.”
  62. 62. Friction eme = rv(of+i) – (f+a) Test Protocol ID: TP2089 1 Length But This Not this 21.2% Increase in clicks
  63. 63. Friction eme = rv(of+i) – (f+a) Test Protocol ID: TP2081 1 Length But This Not this 41.4% Increase in clicks 52.5% Increase in app starts
  64. 64. Friction eme = rv(of+i) – (f+a) Test Protocol ID: TP2026 2 Difficulty But this (1 Option) Not this (6 Options) 20% Increase in clicks 46% Increase in purchases
  65. 65. Friction eme = rv(of+i) – (f+a) Test Protocol ID: TP2026 2 Difficulty Not this But This 10% Increase in clicks
  66. 66. Friction eme = rv(of+i) – (f+a) Test Protocol ID: A-TP1002 What else contributed to differences in performance? Not This But This
  67. 67. Friction eme = rv(of+i) – (f+a) Test Protocol ID: A-TP1002 What else contributed to differences in performance? Not This But This
  68. 68. Anxiety (a) Decreasing Perceived Cost
  69. 69. Anxiety eme = rv(of + i) – (f + a) © Anxiety – Psychological concern stimulated by a given (or unaddressed) element in the email process
  70. 70. Anxiety eme = rv(of+i) – (f+a) They’re just hard water stains…
  71. 71. Anxiety F eme = rv(of+i) – (f+a) Key Principles • Anxiety is just as lethal to conversion as friction. • Try and mitigate anxiety rationally and you are likely to fail… • Though created from legitimate concern, its impact in the mind is often disproportionate to the reality of risk. • While website visitors are skeptical and cautious, the natural level of concern is even greater for email offer recipients.
  72. 72. Experiment  eme = rv(of+i) – (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2083 Background: A large event management online software provider. Goal: To increase total lead inquiries (phone calls + form sign-ups) from visitors who abandoned the free trial sign-up process. Primary research question: Which email tone will result in a higher rate of lead inquiries? Approach: A/B single factorial split
  73. 73. Experiment Control Subject Line: Your Free RegOnline Access Dear [Name], You’re just one step away from getting FREE access to RegOnline, our award winning Event Registration and Management Software. Quickly make an event website, try our event marketing tools, build a registration form template or even generate custom name badges. Click here to finish your profile and get started. Your personal profile is kept secure and we promise NEVER to sell or misuse your information. Have questions? Call us direct at 1-800-XXX-XXXX -- we can even set up your free access over the phone. Sincerely, Jon Powell Customer Service Representative RegOnline Direct: 1-800-XXX-XXXX ext. XXXX jon.powell@RegOnline.com • The control email uses a typical sales tone of a direct-response email.
  74. 74. Treatment Subject Line: Your Free RegOnline Access Hi [First], I noticed that you started the process of getting free access to RegOnline but weren’t able to finish. Are you concerned about giving out your phone number? Are you worried about high pressure sales tactics or mandatory contracts? We believe our product sells itself, so we’re just here to provide you with whatever assistance you need in getting your event up and running - in whatever way works best for you. We promise NEVER to sell or misuse your information. Call me direct at 1-800-XXX-XXXX and I can help get you rolling. If you’d rather just try again online, use this link instead. Thank you in advance for your trust! Sincerely, [Name of Representative] Customer Service Representative RegOnline Direct: 1-800-XXX-XXXX ext. XXXX [name]@RegOnline.com • The treatment directly addresses anxiety, taking on a categorical shift in tone – from a directresponse tone to a customer service tone.
  75. 75. Experiment Control Direct Response Tone eme = rv(of+i) – (f+a) Treatment Customer Service Tone
  76. 76. 349% Increase in lead inquiry rate The service tone achieved a statistically significant increase in inquiries Email Inquiry Rate Rel. Diff. Control 1.58% -- Treatment 7.08% 349.41%
  77. 77. Experiment  eme = rv(of+i) – (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2115 Background: National B2C, B2B financial company with an online and retail presence Goal: To increase account activity, engagement Primary research question: Which subject line will generate the most opens and clicks? Approach: A/B/C/D split test (variable cluster)
  78. 78. Experiment # ID1: ID2: ID3: ID4: ID5: ID6: ID7: ID8: ID9: ID10: ID11: ID12: ID13: eme = rv(of+i) – (f+a) Subject Line Tested [Name], Your Account Information is Ready To View Save Time by Paying Your Bills Online Did you forget your [Bank Name] Online username or password? Manage your money more effectively with [Bank Name] Online Your Account Information is Ready to View Check your account balance online, anytime. Sign on now. Sign on to your [Bank Name] checking account It's easy to log-in to your [Bank Name] account online Sign on to your checking account Check your account balance online, anytime Easily manage your accounts in one place with [Bank Name] Online Don't forget to access your [Bank Name] account online Save time managing your money with [Bank Name] Online
  79. 79. Experiment # ID1: ID2: ID3: ID4: ID5: ID6: ID7: ID8: ID9: ID10: ID11: ID12: ID13: eme = rv(of+i) – (f+a) Subject Line Tested Open [Name], Your Account Information is Ready To View 30.77% Save Time by Paying Your Bills Online 30.70% Did you forget your [Bank Name] Online username or password? 28.49% Manage your money more effectively with [Bank Name] Online 28.21% Your Account Information is Ready to View 26.97% Check your account balance online, anytime. Sign on now. 25.23% Sign on to your [Bank Name] checking account 25.21% It's easy to log-in to your [Bank Name] account online 24.43% Sign on to your checking account 21.26% Check your account balance online, anytime 20.44% Easily manage your accounts in one place with [Bank Name] Online19.32% Don't forget to access your [Bank Name] account online 19.27% Save time managing your money with [Bank Name] Online 16.53%
  80. 80. Experiment # ID1: ID2: ID3: ID4: ID5: ID6: ID7: ID8: ID9: ID10: ID11: ID12: ID13: eme = rv(of+i) – (f+a) Subject Line Tested Open [Name], Your Account Information is Ready To View 30.77% Save Time by Paying Your Bills Online 30.70% Did you forget your [Bank Name] Online username or password? 28.49% Manage your money more effectively with [Bank Name] Online 28.21% Your Account Information is Ready to View 26.97% Check your account balance online, anytime. Sign on now. 25.23% Sign on to your [Bank Name] checking account 25.21% It's easy to log-in to your [Bank Name] account online 24.43% Sign on to your checking account 21.26% Check your account balance online, anytime 20.44% Easily manage your accounts in one place with [Bank Name] Online19.32% Don't forget to access your [Bank Name] account online 19.27% Save time managing your money with [Bank Name] Online 16.53%
  81. 81. Experiment # ID3: ID1: ID9: ID5: ID7: ID12: ID10: ID6: ID8: ID11: ID4: ID13: ID2: eme = rv(of+i) – (f+a) Subject Line Tested Click Open Did you forget your [Bank Name] Online username or password? 30.77% 4.59% [Name], Your Account Information is Ready To View 4.44% 30.70% Sign on to your checking account 4.40% 28.49% Your Account Information is Ready to View 4.34% 28.21% Sign on to your [Bank Name] checking account 3.84% 26.97% Don't forget to access your [Bank Name] account online 3.64% 25.23% Check your account balance online, anytime 3.59% 25.21% Check your account balance online, anytime. Sign on now. 3.47% 24.43% 2.90% It's easy to log-in to your [Bank Name] account online 21.26% Easily manage your accounts in one place with [Bank Name] Online20.44% 1.94% Manage your money more effectively with [Bank Name] Online 19.32% 1.70% Save time managing your money with [Bank Name] Online 1.54% 19.27% 1.05% Save Time by Paying Your Bills Online 16.53%
  82. 82. Live Optimization Email Messaging Effectiveness
  83. 83. Live Optimization  eme = rv(of+i) – (f+a) Partner ID: The New York Times Location: MarketingExperiments Research Library Test Protocol Number: TP2802 Background: Internationally recognized news service known for their journalism. Our Goal: To significantly increase subscriptions. Primary research question: Which email will generate the greatest subscription rate? Approach: A/B split test (variable cluster)
  84. 84. Live Optimization eme = rv(of+i) – (f+a) The Subscription Product Offering A subscription to the NY Times • Home delivery of the Print edition • Unlimited, unrestricted access to articles online, in their smartphone and tablet apps • Free digital access for one family member or friend • Subscriber only special offers, discounts, invitations Incentive • 50% off for the first 12 weeks
  85. 85. The List
  86. 86. THE CONTROL Subject Line: Open this now for Special Savings
  87. 87. BEFORE AFTER
  88. 88. 62% Increase in Conversions The treatment increased the subscription conversion rate by 62.1% Email + Subject Line Open Rate* % Rel. Change Click Rate* % Rel. Change Conv. Rate* % Rel. Change Control 13.50% -- 0.67% -- .008% -- Treatment 15.60% 15.5% 1.01% 99% 50.7% 99% .013% 62.1% *Note: Absolute data points are anonymized, relative differences are actual 96%
  89. 89. Review Email Messaging Effectiveness
  90. 90. Review Pay attention to the change in the mind; not the change in the content eme = rv(of + i) – (f + a) © Perceived Value i Perceived Cost a o f f
  91. 91. Review F eme = rv(of+i) – (f+a) Key Principles • There’s a story behind everything that’s bought. This occurs on two levels: internal and external • It is important to distinguish the difference between the product offer and the click-through offer. • Consider the use of artificial urgency as an incentive to take action • Aggravation in email messaging can occur on two levels: length and difficulty • The natural level of concern in email is strong; do not try to mitigate it rationally.
  92. 92. February 17-20 Aria Las Vegas FLINT MCGLAUGHLIN DAN ARIELY NOAH GOLDSTEIN The longest running research-based email marketing event Plus, brand side marketers and tactical training sessions MECLABS.com/Email
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×