Speaker: Mike Madden, Sr. Manager - Marketing @ Marketo

2. Secrets Flat Out Stink –
Let’s Reveal Marketo’s Secrets
to Demand Generation
Mike Madden

4. Agenda
What is A/B
Testing?
1
Testing
Methodology
2
Marketo’s
Demand
Gen Secrets
3
Conclusion
4

5. Why A/B Testing is Critically Important
A/B testing increases
engagement with
your buyers
A/B testing enhances
campaign
effectiveness
A/B testing
makes you a
better
marketer

6. Marketo’s
A/B Testing
Methodology
1. Choose
one element
to test
2. Ask a
question / write
a hypothesis
3. Decide on the
sample group
4. Define what
success looks
like
5. Set up the
test
6. Analyze the
test results
7. Determine
the winner
8. Implement
the new
control

7. Use Statistical Significance Calculators!

8. A/B Testing
Email Send Time

9. What’s the best time of day to send
marketing emails?

10. The Time Test Set Up
Total Audience
All marketable
names, last
engagement date in
past 1 year,
marketing job titles
5am (Control)
10am
1pm
Details:
• Split total audience 3 ways
• Backed into numbers to see at least
1,000 opens per test (for sig.)
• Sent each email from a different
campaign to ensure exact timing on
the send
• Tested the time testing over 7
different email campaigns
• Over 200,000 emails sent
• Tested in early, mid, and late buying
stages

11. The Time Test Set Up
Email Sent Delivered % Delivered Opens % Opened Clicks CTR CTO Unsub. Unsub %
5am Control 70,128 68,791 98.1% 5983 8.7% 306 0.4% 5.1% 98 0.14%
10am 68,034 66,763 98.1% 6227 9.3% 342 0.5% 5.5% 112 0.17%
1pm 68,067 66,754 98.1% 6642 9.9% 356 0.5% 5.4% 102 0.15%
20% higher
click
through
rates
5% higher
click to
open rates
14% higher
open rates
1PM IS THE WINNER

12. Your Website
Using Triggers to Drive Revenue

13. Why use triggers? Because they drive 3x
more engagement than batch/nurture
emails

20. Ex: Email Product Page Campaign Flow
Wait 4
Minutes
If a known
subscriber
visits the
Email
Product
Page…
Organic Behavior Segmentation Action – Send Email
And that subscriber is:
• Marketable
• Target Status (Target)
• Not was sent any
email in past 30
minutes
• Meets vertical specific
criteria

21. Ex: Email Product Page Campaign Flow

22. Trigger Results vs. Standard Batch / Nurture Emails
7,675% more
efficient at
generating an
opportunity
261%
higher open
rates
157%
higher click
to open
rates 833%
higher click
through
rates

23. Content Strategy
Who Likes What, When and Why

24. Our MQL Model
𝑀𝑄𝐿 = 𝑇𝑎𝑟𝑔𝑒𝑡 𝑆𝑡𝑎𝑡𝑢𝑠 (𝑇𝑎𝑟𝑔𝑒𝑡) + 𝐵𝑒ℎ𝑎𝑣𝑖𝑜𝑟 𝑆𝑐𝑜𝑟𝑒 (> 19) + 𝐴𝑐𝑐𝑜𝑢𝑛𝑡 𝑆𝑐𝑜𝑟𝑒 (> 29)
• Right Job Title
• First/Last Name
• Valid Email/Phone
• Company Name
• Email Clicks
• Form Completions
• Visited Booth
• Attended Webinar
• Attended Demo
• Downloaded Analyst
Report
• Scoring decay to
capture inactivity
Predictive Account
Score powered by
Leadspace
Looks at:
• Annual Revenue
• Employee Size
• Industry
• Location
• Technologies Used
• Other signals like
hiring trends, funding
levels, social media
activity, etc.
Job Title/Function
Demo
Score
Marketing/Ops -
Practitioner
30
Marketing/Ops -
Manager
38
Marketing/Ops –
Dir/VP
45
Marketing CMO 50

25. We Have 3 Types of Buyers
Behavior
Score = 0
Early
Stage
Behavior
Score 1-29
Mid
Stage
Behavior
Score > 29
Late
Stage

26. Buyers Plotted Out by Behavior Score
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
90,000
Early Stage Mid Stage Late Stage
Total Audience

27. Who registers for our webinars? Who
attends them?

28. An Early Stage Webinar

29. After the 1st Invite, Here’s Who Registered…
5
185
1037
0
200
400
600
800
1,000
1,200
Early Stage Mid Stage Late Stage
Total Audience

30. Another Early Stage Webinar

31. And Yet We See It Again!
155
687
2716
0
500
1,000
1,500
2,000
2,500
3,000
Early Stage Mid Stage Late Stage
Total Registered Leads
Total Audience

32. Here’s the 8 Biggest Mistakes Attendee List
23
184
889
0
100
200
300
400
500
600
700
800
900
1,000
Early Stage Mid Stage Late Stage
Total Attended Leads
Total Audience

33. 8 Biggest Mistakes Attendance Rates
15%
27%
33%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Early Stage Mid Stage Late Stage
Attendance Rates

34. Let’s go back to first email invite for the
3 Hacks to Boost Open Rates webinar…

35. Mid/Late Was Better Than Early Stage
131% higher
open rates
1,320%
higher click
through
rates
984% higher
click to open
rates
47% lower
unsubscribe
rates
8,965%
higher
landing page
conversion
rates!

36. Final 3 Hacks Webinar Breakdown
207
726
1745
53
253
544
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
Early Stage Mid Stage Late Stage
Registered vs. Attended by Stage
Registered
Attended
26%
attendance
35%
attendance
31%
attendance

37. What We Learned
 Content is all about investment
 Give people what they want, when they want it

38. Operations
How to Use Triggers to Boost Email Deliverability

39. Continuously Bouncing Emails Are REAL

40. Sender Reputation
Sender
Reputation
Unique
Clicks
Open Rate
Positive
Engagements
Spam
Complaints
& Spam Trap
Hits
Hard
Bounces
Blacklists
Negative
Engagements

41. Soft Bounce Management Campaigns
Soft Bounce: A soft bounce is a temporary problem with email
deliverability, usually due to an unavailable server or a full inbox.
We created two campaigns:
 Batch clean up: For any email that has soft bounced a minimum number of
10 times in the past 90 days, mark it as invalid.
 Trigger clean up: For any email that soft bounces a minimum number of 6
times in the past 30 days, mark it as invalid.

42. Bounce Type Audit

43. Bounce Category Management
We built campaigns to clean up specific soft bounce categories that
could be detrimental to our sender reputation in the future.
We created two more campaigns:
 Batch clean up: Scrub all existing emails that have poor soft bounce
categories (3, 4, and 9)
 Trigger clean up: Clean up poor soft bounce categories in real time

44. Deliverability vs. Open Rates
10%
11%
12%
13%
14%
15%
16%
17%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
100%
Jan 2016 – Sep 2016
% Delivered
% Opened

45. Deliverability vs. Open Rates
10%
11%
12%
13%
14%
15%
16%
17%
18%
90%
91%
92%
93%
94%
95%
96%
97%
98%
99%
100%
Jan 2016 – Jun 2017
% Delivered
% Opened
Bounce
Mgmt.

46. What We Learned
 There is a direct correlation between deliverability rates and open
rates
 Deliverability rates went from 93% -> 99%, which boosted open
rates from 13.5% -> 17.3%!
 Something as simple as email deliverability can be an ENORMOUS
revenue driver!