Taking your marketing performance to the next level requires pulling all the pieces together to get greater clarity on performance, within and across channels. This is where an advanced attribution solution can be of help. Learn how the UK’s leading price comparison website, Moneysupermarket.com, plans to leverage these capabilities to drive big results.
Pulling together your cross channel marketing pieces
1. Click to edit Master title style
• Pulling Togetherstyles
Click to edit Master text Your
–
Cross-Channel
Second level
Marketing Pieces Via Advanced
• Third level
Attribution
– Fourth level
» Fifth level
Casey K Carey
Chief Marketing Officer – Adometry
2. Click to edit Master title style
pulling together your
• Click to edit Master text styles
CROSS-CHANNEL
MARKETING
– Second level
• Third level
– Fourth level
» Fifth level
via advanced attribution
2
#Engage13
3. a bit about us
Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
Casey Carey
Chief Marketing Officer
@caseycarey
Ben Sidebottom
Head of Media Systems
#Engage13
4. to succeed, modern marketers must
increasingly…
Click to edit Master title style
• Click to edit Master text styles
–Become level
Second
data-driven,
• Third level
Understand the
customer journey,
Be multi-channel in
approach,
but it has become
more complex.
but organize and
measure in silos.
– Fourth level
» Fifth level
but there is a
data deluge.
#Engage13
5. Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
#Engage13
6. Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
last-click, the attribution
zombie
7. contrary to popular belief,
last-click is the living dead
Click to edit Master title style
• Click to edit Master text styles
– Second level
Digital makes it possible
to measure everylevel
• Third touchpoint
– Fourth level
Attribution makes » Fifth level
the
connection to ROMI
54%
14%
42%
of marketers use last click
attribution…
consider last-click to be “very
effective”…
are unsure which attribution
approach they should use…
Source: eConsultancy, Marketing Attribution, Valuing the Customer Journey, 2012
#Engage13
8. Click to edit Master title style
LAST-CLICK ATTRIBUTION
• Click to edit Master text styles
•
•
•
•
– Second level
• Third MYOPIA
CHANNEL level
– Fourth level
INEFFICIENT SPEND
» Fifth level
LOWER PERFORMANCE
UNREALIZED POTENTIAL
9. Using advanced attribution to…
Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
…understand the true
connection between marketing
and results.
9
10. Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
#Engage13
#Engage13
11. descriptive model approach to advanced
attribution
Click to edit Master title style
“Data Driven A/B
• Click to edit Master text styles Testing”
– Second level
• Third level
– Fourth level
» Fifth level
10.0%
Conversion
Rate
6.5%
Conversion
Rate
#Engage13
12. Click to edit Master title style
10.0%
• Click to edit Master text styles
Conversion
Rate
– Second level
Channel
–
Campaign
Fourth level
+ Recency
» + Frequency
Fifth level
+ Recency
Site
+ Frequency
+ Recency
Placement
+ Frequency
+ Recency
Creative
+ Frequency
+ Recency
Specificity
Confidence
• Third level
+ Frequency
Hierarchical Bayesian
Shrinkage
•
A principled statistical approach
to data sparsity
•
Each weight is derived from
confidence weighted average
of weights at different levels of
granularity
#Engage13
13. providing insights and clarity to crosschannel performance
Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
Attribution
– Fourth level
» Fifth level
•
•
•
•
•
•
•
KPIs
Conversion Events
Channel
Source
Campaign
Placement, keyword, etc.
Creative, content, format
Reach & Frequency
Cross-Channel
Dashboard
14. using attribution to fuel predictive
optimization
Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
Past
– Fourth level
» Fifth level
Future
Present
Attribution
Optimization
How did my marketing perform?
How can we improve?
#Engage13
15. accounting for diminishing returns
nature title style
Click to edit Masterof marketing
Maximum
Marginal
Return
Maximum
Average
Return
• Click to edit Master text styles
– Second level
Point of
Saturation
KPI Performance
• Third level
– Fourth level
» Fifth level
Ad Volume
#Engage13
16. optimal investment across and within
channels
Click to edit Master title style
Evaluating Scenarios
• Overall Optimization Master text styles
Click to edit
For Each Scenario –
Specific Recommendations
– Second level
• Third level
– Fourth level
» Fifth level
View optimization
opportunities across
channels
Reallocate spend
across and within
channels
Granular line item changes – “I/O
ready”
• Modify default recommendations
• Rerun prediction
• Evaluate revised KPIs
#Engage13
17. driving greater ROMI
Click to edit Master title style
Value Drivers
• Click to edit Master text styles
Display eCPA
– Second level
• Third level
Optimization Within Channels
– Fourth level
Areas of Impact
20% - 30% decrease in effective CPA for display and
retargeting
10% - 20% improvement by optimizing channel
performance including PPC, Affiliate, Email, and Social
» Fifth level
Optimization Across Channels
20% - 40% improvement in performance by optimizing
spend across various channels
Reduction in Analysis and Reporting Costs
25% - 50% decrease in effort required to pull, aggregate,
and report on marketing performance across channels
Overall Marketing ROI
20% - 40% improvement in overall performance of
marketing investments
#Engage13
18. Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
Adometry
and MSM
Nov
2013
19. Overview of MSM
Click to edit Master title style
• One of the UK’s largest
• price comparison websitesstyles
Click to edit Master text
– Second level
• £204m Third level
• annual revenue
– Fourth level
» Fifth level
• In-house digital team of 50+
• Digital transformation
programme
19
20. Why change our view?
Click to edit Master title style
Display Impression
Organic Click
• Click to edit Master text styles
Paid Search Click
– Second level
• Third level
– Fourth level
» Fifth level
Direct To Site
Happy Customer!
20
#Engage13
21. How we changed our view?
Arbitrary Models
Click to edit Master title style
• Click to edit Master text styles
50%
– Second level
Last Click
First Click
Even Click
Last/First
Click
U Shape
Model
Time Decay
Model
Programmatic
(Self Defining
Model)
• Third level
Order of complexity
– Fourth level
» Fifth level
21
#Engage13
22. Challenges in Changing Our Approach
Click to edit Master title style
Business Acceptance
Tagging
Calibration and data
integration
• Click to edit Master text styles
– Second level
•
Lack of understanding
•
Replacing•current trusted
Third level
model
– Fourth level
•
Digital marketing
optimisation
•
Financial reporting
•
Change management
•
» Fifth level
Code implementation /
Resource
•
Iterative process
•
•
Defining click and
impression based touch
points
Standardising reporting
parameters
•
Rule testing
•
Linking data sources
•
Process changes
•
UI reporting and data feeds
22
#Engage13
23. Opportunities
Today
Click to edit Master title style
Future
• Click to edit Master text styles
– Second level
TV /
• Third level
Econometric
Paid
Search
Modelling
– Fourth level
» Fifth level
Audience+
Partner Portal
23
#Engage13
24. Click to edit Master title style QUESTIONS?
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
25. Click to edit Master title style
• Click to edit Master text styles
– Second level
• Third level
– Fourth level
» Fifth level
Casey Carey
casey.carey@adometry.com
Ben Sidebottom
ben.sidebottom@moneysupermarket.com
#Engage13
Editor's Notes
To understand the weight or credit due to an event in a particular sequence, we first find all examples of that sequence – both converting and non-converting. From this we can calculate the conversion rate for that sequence.Next, we find a similar sequence of events that excludes one of the events from the prior sequence – for example, the first event. Again, we find both converting and non-converting examples. We then calculate the conversion rate for this (second) sequence.Now, by comparing the conversion rates of the sequences we can determine the weight or impact of the the missing event. If the rates are identical or similar then the missing event deserves little or no credit whereas if the rates are different then that missing event had an impact and deserves credit.There are some points that you can make after this – the death is in the details stuff:Obviously simplified explanation.Number of sequence and length of sequences will varyWe normalize the weights such that they add up to one (1)Tons of events - "big data problem"If there is only one event in the sequence it will get 100% of the creditAn event is actually a complicated thing – has lots of attributes – site, campaign, placement, creative concept, tactic, etc. we calculate at the lowest level and roll the results upAlso frequency and recency.But this is the stuff that the engineers have all worked out in a principled manner. Machine learning, etc.Data Driven Algorithm ConfigurationScientific approach to determining Look back & Look ahead periodsDealing with Data SparsityBayesian Hierarchical Shrinkage Absolute vs. Relative AttributionAccounting for baseline behaviorCausality vs. CorrelationInverse propensity weighting
Lays the foundation for our benefits.Introduction to how we approach the problem of looking at historical data and making changes to improve performance.
Introduction and agenda - caseyThe business case for better attribution - caseyHow advanced attribution works - casey Data requiredModeling methodologyTypes of reportsOptimizationOverview of MSM - benWhy an advanced attribution initiative (objectives of initiative) - benExpected opportunities to leverage new insights to improve performance across channels - benOpportunities for integrating marketing systems and data sources to provide more accurate attribution and leverage the result in marketing channels –ben Q&A – casey & ben
- Overview of MSM - simple
Take a typical path of a user with 4 different touch points; user searches for car insurance and clicks on a MSM PPC ad then bookmarks and decides to come back later. They then return via that bookmark and we pick up a DTS click. The user fillss in the form and tells us he is due for a renewal and gets some quotes but doesn’t complete the purchase. MSM then find him later on another site and deliver a customised ad about his car insurance renewal. He sees the ad and types Moneysupermarket into his browser search and clicks on an organic click. The user then completes the quote and purchases a car insurance policy.You can see a last click model would simply say organic gets all of the revenue. If 100% of our users did this exact touch point journey, last click would tell us to switch off all PPC and Display as all revenue would go to organicWhen we look at this we know other medias took part in getting the user to convert and it wasn’t all of the work of organic. We know that is not an efficient way to do things and that the PPC click and Display impression had an influence on the users final purchase
MSM used a last click model for 100% of the revenue share in both digital media and financial reporting. Now, MSM have adopted a new model and we have started applying it across the channels to gain insights and efficiencies. 18 months ago, we started the process of reviewing our attribution model and looked for ways we could gain efficiencies from our marketing spends and also how we can diversify our marketing mix by bringing in new medias which were previously stopped due to poor performance on a last click basis.We reviewed the process and started by looking at the available models (start with arbitrary models)Explain each model briefly but end with all models are arbitrary. Also these models do not take into account non-converting paths – no down weightingShow the meaning of ‘arbitrary’ and explain that picking any of the first models is simply picking one by random choice with no proof that it works for their businessShow programmatic model at the end then outline it is more complex the further up the ordering you go
MSM faced many challenges and still do when it comes to remodelling the data. The 3 main areas of challenge are getting business acceptance, tagging the site correctly and calibration and data integration.Business acceptance – biggest challenge here is getting people to understand the need for a new approach and how the new approach works. This also relates to replacing a trusted process. What people do not understand, they tend to not trust; which makes sense, we are humans. So getting people involved early and explaining the modelling systems throughout the process helped get people on-board early and once you have a few on-board it helps get others on too. Digital marketing optimisation was always done on last click and it worked. Channels perform and the optimisation efforts to date were all based off last click data so changing this is also a challenge as it effects a large portion of our digital marketing team. Financial reporting is the same, processes in place and it is what budgets and marketing efforts are reported on.Tagging – To implement such a complex tool correctly, you need to have resource available and committed to the project. At MSM, we did have a dedicate development resource which made things easier but there are challenges around getting the dev team to understand the benefits or what the tag is doing. Standardising the reporting parameters was probably our biggest challenge. We had to go into each channel and understand what parameters we currently use in our URLs or pages and create a mapping table. We use a common source code in our internal tracking, but we didn’t find that granular enough to do what we needed, so a new parameter structure was constructed and deployed across the medias. Testing as well needs to be robust and thorough. Even though the tag itself was pretty simple, the different channels, combinations, entry points and user behaviours (like clicking back button or going from secure to non-secure which doesn’t pass through the referring URL) all proved to be trickier than first thought and we had to iteratively test and review the implementation to make sure it passed all tagging requirements.Calibration – so calibration was done on the reporting side to make sure that the data was being reported on correctly and that the collection of data was as accurate as possible. This was an iterative process with many tweaks to the data points. Examples of this is where we didn’t capture the RTB bid ID in the creative tag so we had to add this and let the data collect. Defining click and impression based touch points was something we have customised with our instance of attribution. So paid search is obvious with click based but when it comes to display, we have customised some bits there. Some activity we run is paid on a CPC so providing data based on impression touch points didn’t make sense and we found it was putting the algorithms off a bit as they were optimising to clicks, not impressions. Data linking was an important part of our process. We have a lot of data feeds going in and out of adometry to achieve the goals we have. So going into Adometry we have the adometry tag data, our conversion feed, marin ppc feed, mediamath impression feed, media partners click feed – for our external partners we work with, shopping feeds as they come from a different source, 3 different lookup tables for campaigns-conversion types and audience+ campaigns to pixels and a channel conversion aggregation lookup. Then UI reporting was going through the different reporting dimensions and combinations and making sure we had all of the reports that we needed to be able to do our insights and optimisation.
- Will go through this off the cuff. Will talk through what we hope to gain from each area and how it will work. Will be quite a lot of talking in each area but will keep it within the time limits.