Predictive Content
Introduction
Sept, 2016
Page 2Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Overview
1. What is Predictive Content
2. How it works
3. Recommendation Algorithms
4. Case studies and real-life examples
5. How Marketo uses Predictive Content
Page 3Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Intro
• Listening everywhere and communicating anywhere powered by
machine learning empowers marketers to maximize bottom line
results leveraging marketing assets and audience data
• Automatically recommend the right content to the right prospect
in a scalable way driven by data
• Self learning engine designed for rocket scientists marketers
Page 4Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Background
• Marketers drive thousands of customers to their web site
• SMB’s: 10 - 50K visitors per month
• Enterprises: 100 - 500K visitors per month
• B2C’s: 100K - over 1 million visitors per month
• They also create hundreds of content assets to engage more effectively
• Average SMB has +290 content assets
• Average Enterprise has thousands
• White papers, videos, case studies, blog posts, special offers
Page 5Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Engagement (R)evolution
• Rule based targeting
• Great for small # of events, content products and verticals
• A/B testing
• Further optimizing multiple engagement options for
segmented audience
• Predictive
• A scalable way to auto-engage with the most relevant content
to each individual customer driven by data, powered by
machine learning
Page 6Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Predictive Content
• Enables marketers to optimize engagements and conversions in
every online interaction
• Automatically presents the most relevant content for each
individual visitor
• Based on machine learning algorithms and predictive analytics
Page 7Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
How it Works
1. Auto-Discovers and maps all of your content assets
2. Learns which content works best and for who (training/testing)
3. Recommends relevant content to web visitors & leads
Increases content consumption and onsite engagement
Page 8Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
• Auto-discover your content
• Analyze and track behavior to discover assets
DISCOVER LEARN RECOMMEND
Page 9Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Content Discovery
Page 10Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Content Discovery
Page 11Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
• Record actual content consumption
• Analyze performance for different
audiences
• Run algorithms that learn what works
best for who
DISCOVER LEARN RECOMMEND
Machine Learning
Page 12Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Algorithm Types
• Classification Algorithms (Logistic regression, Boosted decision Trees)
• Probabilistic algorithms Markov-chain
• Interest graphs
Page 13Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
The models are tuned to..
1. Detect successful and unsuccessful content engagements
2. Identify visitor/lead attributes in real-time
3. Discover positive correlations that could predict future engagements
4. Continuously optimize recommendations (every 8 – 24h)
5. Expose NEW content, detect trending and tune down old content
Page 14Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
• Identify web visitor attributes
• Recommend most relevant content
• Optimize engagements
DISCOVER LEARN RECOMMEND
Page 15Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Example – Web Recommendation Bar
Page 16Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Example – Web Recommendation Bar
Page 17Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Example – Web Rich Media
Page 18Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Customer Example (Web Rich Media)
Page 19Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Customer Example
• 75% direct lead conversion rate
from predictive content
recommendation engine.
• Generated $100K in
opportunities
Page 20Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Customer Example (Web Rich Media)
Page 21Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Customer Example (Web Rich Media)
Page 22Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Cross Channels
Web Mobile Email
Page 23Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Content Recommendation Results
315%Increase in site engagements
(pages per visit, time on site) for
targeted visitors
201%Increase in content specific lead
conversion
3xMore views on recommended content
compared to generic views
18%Increase in content consumption
Page 24Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 25Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 26Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 27Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 28Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
1. Enable Content Discovery
Page 29Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
2. Select Your Content
Page 30Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
2. Select Your Content
Page 31Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
2. Select Your Content
Page 32Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
2. Select Your Content
Advanced: use categories
Page 33Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
3. Add Some Code
rtp('send','view');
rtp('get','rcmd', 'richmedia');
<div class="RTP_RCMD2" data-rtp-template-id="template1"></div>
Add 2 lines to your standard RTP snippet:
Add a line of code where you want the recommendations to appear on your page:
Page 34Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 35Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 36Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Page 37Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
A Measurable Lift in Downloads
New website launch
Recommendations added
Page 38Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Q4 – Predictive Content in Email
*not final
Page 39Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Q4 – Predictive Content in Email
*not final
Page 40Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Q4 – Predictive Content in Email
*not final
Page 41Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Q4 – Predictive Content in Email
*not final
Page 42Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Q4 – Predictive Content in Email
*not final
Page 43Marketo Proprietary and Confidential | © Marketo, Inc. 9/28/2016
Predictive Content - Summary
• Enables marketers to optimize engagements and conversions
• Automatically recommends the most relevant content
for each individual visitor
• Based on machine learning algorithms and predictive
analytics
Thank you.

Marketo Predictive content

  • 1.
  • 2.
    Page 2Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Overview 1. What is Predictive Content 2. How it works 3. Recommendation Algorithms 4. Case studies and real-life examples 5. How Marketo uses Predictive Content
  • 3.
    Page 3Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Intro • Listening everywhere and communicating anywhere powered by machine learning empowers marketers to maximize bottom line results leveraging marketing assets and audience data • Automatically recommend the right content to the right prospect in a scalable way driven by data • Self learning engine designed for rocket scientists marketers
  • 4.
    Page 4Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Background • Marketers drive thousands of customers to their web site • SMB’s: 10 - 50K visitors per month • Enterprises: 100 - 500K visitors per month • B2C’s: 100K - over 1 million visitors per month • They also create hundreds of content assets to engage more effectively • Average SMB has +290 content assets • Average Enterprise has thousands • White papers, videos, case studies, blog posts, special offers
  • 5.
    Page 5Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Engagement (R)evolution • Rule based targeting • Great for small # of events, content products and verticals • A/B testing • Further optimizing multiple engagement options for segmented audience • Predictive • A scalable way to auto-engage with the most relevant content to each individual customer driven by data, powered by machine learning
  • 6.
    Page 6Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Predictive Content • Enables marketers to optimize engagements and conversions in every online interaction • Automatically presents the most relevant content for each individual visitor • Based on machine learning algorithms and predictive analytics
  • 7.
    Page 7Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 How it Works 1. Auto-Discovers and maps all of your content assets 2. Learns which content works best and for who (training/testing) 3. Recommends relevant content to web visitors & leads Increases content consumption and onsite engagement
  • 8.
    Page 8Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 • Auto-discover your content • Analyze and track behavior to discover assets DISCOVER LEARN RECOMMEND
  • 9.
    Page 9Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Content Discovery
  • 10.
    Page 10Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Content Discovery
  • 11.
    Page 11Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 • Record actual content consumption • Analyze performance for different audiences • Run algorithms that learn what works best for who DISCOVER LEARN RECOMMEND Machine Learning
  • 12.
    Page 12Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Algorithm Types • Classification Algorithms (Logistic regression, Boosted decision Trees) • Probabilistic algorithms Markov-chain • Interest graphs
  • 13.
    Page 13Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 The models are tuned to.. 1. Detect successful and unsuccessful content engagements 2. Identify visitor/lead attributes in real-time 3. Discover positive correlations that could predict future engagements 4. Continuously optimize recommendations (every 8 – 24h) 5. Expose NEW content, detect trending and tune down old content
  • 14.
    Page 14Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 • Identify web visitor attributes • Recommend most relevant content • Optimize engagements DISCOVER LEARN RECOMMEND
  • 15.
    Page 15Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Example – Web Recommendation Bar
  • 16.
    Page 16Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Example – Web Recommendation Bar
  • 17.
    Page 17Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Example – Web Rich Media
  • 18.
    Page 18Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Customer Example (Web Rich Media)
  • 19.
    Page 19Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Customer Example • 75% direct lead conversion rate from predictive content recommendation engine. • Generated $100K in opportunities
  • 20.
    Page 20Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Customer Example (Web Rich Media)
  • 21.
    Page 21Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Customer Example (Web Rich Media)
  • 22.
    Page 22Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Cross Channels Web Mobile Email
  • 23.
    Page 23Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Content Recommendation Results 315%Increase in site engagements (pages per visit, time on site) for targeted visitors 201%Increase in content specific lead conversion 3xMore views on recommended content compared to generic views 18%Increase in content consumption
  • 24.
    Page 24Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 25.
    Page 25Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 26.
    Page 26Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 27.
    Page 27Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 28.
    Page 28Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 1. Enable Content Discovery
  • 29.
    Page 29Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 2. Select Your Content
  • 30.
    Page 30Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 2. Select Your Content
  • 31.
    Page 31Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 2. Select Your Content
  • 32.
    Page 32Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 2. Select Your Content Advanced: use categories
  • 33.
    Page 33Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 3. Add Some Code rtp('send','view'); rtp('get','rcmd', 'richmedia'); <div class="RTP_RCMD2" data-rtp-template-id="template1"></div> Add 2 lines to your standard RTP snippet: Add a line of code where you want the recommendations to appear on your page:
  • 34.
    Page 34Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 35.
    Page 35Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 36.
    Page 36Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016
  • 37.
    Page 37Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 A Measurable Lift in Downloads New website launch Recommendations added
  • 38.
    Page 38Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Q4 – Predictive Content in Email *not final
  • 39.
    Page 39Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Q4 – Predictive Content in Email *not final
  • 40.
    Page 40Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Q4 – Predictive Content in Email *not final
  • 41.
    Page 41Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Q4 – Predictive Content in Email *not final
  • 42.
    Page 42Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Q4 – Predictive Content in Email *not final
  • 43.
    Page 43Marketo Proprietaryand Confidential | © Marketo, Inc. 9/28/2016 Predictive Content - Summary • Enables marketers to optimize engagements and conversions • Automatically recommends the most relevant content for each individual visitor • Based on machine learning algorithms and predictive analytics
  • 44.

Editor's Notes

  • #2 Update with new cover from visual brand deck.
  • #13 Algos in dev Classification algo’s: (trying to classify which audience will create a click and which won't for each asset that might be presented to it and at what probability) Logistic regression - in dev. based on attribute vectors. decisions trees - were using boosted trees - in dev. boosted Bar uses Markov, SFO and metadat Markov-style algo (looks one step back): Item-based. based on the last seen asset. finds assets that have the strongest connections between them Collaborative filtering (sfo) - looks at the asset clicks, grades them based on key attributes (country, state, industry, search term, visit #). then , when a visitor comes in we find the asset with the highest combined grade for the visitors attributes. highest for sum of grade for location, industry rule based - user attributes vs asset attributes. this is the fallback rich media new (discovery date), trending (14 day slope) , popular (forever)
  • #25 Now lets talk about how we’re drinking our own champagne at Marketo
  • #26 We have the content recommendation bar on our blog right now And with the release of the rich media recommendation functionality we wanted to work that into our new website launch
  • #27 A lot of work was done for the rich media recommendations was done in conjunction with the launch of our new website Objectives: Primary goal is to improve site conversions – as always We also want to improve engagement with our content Our first step was to select and align our exiting content to our new solutions
  • #28 This was the objective of our initial rollout – each solution page has solutions specific asset recommendations To do this we first identified which assets aligned best to each solution From there it was just a matter of configuring the recommendations Now we still have rule based campaign that will be running, such as promoting local events, changing content for different business types, and targeting certain key accounts and visitors. But we also want to compliment those with predictive content recommendations that will run in parallel, require minimum set-up, and yet still be very targeted to individual visitors. Now I’ll show you just went into that process
  • #29 The content discovery feature by itself provides a ton of value and I highly recommend enabling this feature whether or not you utilize content recommendations This will let you see the performance of your content (whitepaper, case studies, blog posts, etc), both from an engagement and lead generation perspective.
  • #34 Your code may look a little different depending on the template you select and any style edits you’d like to make There are many configuration options available to match the style of recommendations to your website’s look and feel
  • #35 Presto! Like magic, your content recommendations are live! From start to finish it took less than 3 hours to get 30+ asset recommendations live on our website Granted we had all the images done already but it was very easy to get going Both the core RTP code and the recommendations code were implemented through our tag management system The container div I added myself through our CMS. That was it!
  • #36 We have several ways of tracking the impact and performance of our recommendations. Right on the recommendations screen we can see the clicks, direct conversions (if there is a form present) and assisted leads. It’s a good practice to have a mix of gated and un-gated assets to recommend so not every will require a form completion – but the great part is that you can still see if a click on that ungated asset led to a conversion later on. This give you a true picture of the value of your content pieces.
  • #37 You can also drill into individual assets for a more detailed look at performance.
  • #38 This trend shows downloads for our Definitive Guide to Digital Marketing Our new website improved the consumption, as was intended But you can see another definite lift once the recommendations were deployed