Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

HeroConf London 2019 - Why your attribution model sucks how to step beyond data-driven models with a markov model approach

513 views

Published on

The choice of the right attribution model is a challenge for most advertisers. We all know the limitation of Last Click attribution model. We will illustrate the limitations of other, more comprehensive pre-defined attribution models such as Time Decay and Position Based or Data Driven. We will share a solution to implement a Markov Chains based approach and will introduce an approach to test a new attribution model.


3 Key Learnings
Overview of attribution models and their limitations
How to implement a Markov Chains
How to test the impact of an attribution model

Published in: Marketing
  • Be the first to comment

HeroConf London 2019 - Why your attribution model sucks how to step beyond data-driven models with a markov model approach

  1. 1. Why Your Attribution Model Sucks How to step beyond Data-Driven Models with a Markov Model Approach Gianluca Binelli | @ktzstyle Most Innovative Presentation HeroConf 2019
  2. 2. Session Outcomes ● Overview of attribution models and their limitations ● How to implement a Markov Chains based model ● How to test the impact of an attribution model
  3. 3. What do we do?
  4. 4. What do we do? Scientific Performance Marketing Agency
  5. 5. Who am I? ● 14y experience in online advertising ● 6.5y at Google between Dublin, NYC & London ● Managed Quarterly $XXM in advertising for Google as part of the SEM in-house team that promotes Google’s products (in FB, Bing, Google, Linkedin etc) ● Advisor for Google’s own equity fund Capital G
  6. 6. What got Kermit drunk?
  7. 7. It all started with an appletini
  8. 8. It all started with an appletini
  9. 9. It all started with an appletini
  10. 10. It all started with an appletini
  11. 11. It all started with an appletini
  12. 12. It all started with an appletini
  13. 13. Was it the champagne?
  14. 14. The smoking gun
  15. 15. Path Length
  16. 16. Conversion path
  17. 17. Multi channel conversion visualiser
  18. 18. Google Ads: Search Attribution
  19. 19. Facebook Attribution
  20. 20. Heuristic Attribution
  21. 21. Like work life balance ● No one size fits all
  22. 22. Let’s assume tracking is working ● Tags/Pixels ● UTMs ● Single source of truth ○ Google Marketing Platform (AKA DoubleClick) ○ Adobe Analytics ○ Facebook Attribution ○ Google Analytics
  23. 23. A few options
  24. 24. Data Driven
  25. 25. Data Driven Shapely Value
  26. 26. Data Driven ● Google Ads ● Google Analytics
  27. 27. Data Driven ● Google Ads ○ 15,000 clicks in 30 days ○ 600 conversions in 30 days ○ Cuts out under 400 conversions per month ● Google Analytics
  28. 28. Data Driven ● Google Ads ○ 15,000 clicks in 30 days ○ 600 conversions in 30 days ○ Cuts out under 400 conversions per month ● Google Analytics 360
  29. 29. Markov Chains The Maths
  30. 30. What’s a chain? ● Let’s assume we have 2 keywords ● These means we have 4 states for each chains: ○ START ○ "OUR BRAND" ○ Advertise Online ○ CONVERSION
  31. 31. What’s a chain? PATH CONVERSIONS START > "OUR BRAND" > Advertise Online > Advertise Online > "OUR BRAND" > "OUR BRAND" > Advertise Online > "OUR BRAND" > CONVERSION 1 START > "OUR BRAND" > Advertise Online > Advertise Online > "OUR BRAND" > CONVERSION 1 START > "OUR BRAND" > "OUR BRAND" > CONVERSION 1 total 3
  32. 32. Once we have all the link counts, we can compute the transition probability ( = the chance to go to a certain state starting from a certain point ) edge Link count Transition probability START > "OUR BRAND" 3 3/3 START > Advertise Online 0 0 TOT START 3 "OUR BRAND" > "OUR BRAND" 2 2/8 "OUR BRAND" > Advertise Online 3 3/8 "OUR BRAND" > CONVERSION 3 3/8 TOT A 8 Advertise Online > "OUR BRAND" 3 3/5 Advertise Online > Advertise Online 2 2/5 Advertise Online > CONVERSION 0 0 TOT B 5 What’s a chain?
  33. 33. Let’s draw something! It is worth noticing that once we are in A and in B we could end up in A and B again. START "OUR BRAND" Advertise Online CONVERSION 100% 25% 60% 37.5% 40% 37.5%
  34. 34. You never realize what you have until it’s gone
  35. 35. You never realize what you have until it’s gone
  36. 36. To assess the impact of a keyword we assume it is gone ● Importance of keyword "OUR BRAND" = the change in conversion rate if keyword "OUR BRAND" is dropped from the Graph ● or in other terms if keyword "OUR BRAND" becomes a NULL state. A NULL state is an absorbing state so if one reaches this STATE can’t move on.
  37. 37. START NULL Advertise Online CONVERSION 100% 25% 60% 37.5% 40% 37.5% On a graph it would look like this
  38. 38. START NULL 100% Which means
  39. 39. START NULL 100% Which means Thus importance of kw "OUR BRAND" (defined as the change in conversion rate) is 1.
  40. 40. START "OUR BRAND" NULL CONVERSION 100% 25% 37.5% 37.5% Let’s do the same for ‘Advertise Online’
  41. 41. START "OUR BRAND" NULL CONVERSION 100% 50% 50% Which means
  42. 42. START "OUR BRAND" NULL CONVERSION 100% 50% 50% Which means As we can see, once we removed kw Advertise Online, the chances to convert fell to .5. Therefore the importance of kw Advertise Online is the difference between previous conversion rate (1) and the chances to convert if we remove Advertise Online (.5), i.e. 0.5.
  43. 43. ● Once we have all the importance weights for all the channels/keywords we can finally compute the number of conversions weighted by the importance of our channels/ keywords. ● We take the previous number of conversions (3 in our example) and we weight them according to the importance weight: ○ 1 / (1+.5) for kw "OUR BRAND" → 3*(⅔) → 2 conversions ○ .5 / (1+.5) for kw Advertise Online → 3*(⅓) → 1 conversion To summarize
  44. 44. Markov Chains The Ingredients
  45. 45. Google Analytics Conversion Paths
  46. 46. Google Analytics Sheets Plugin
  47. 47. Create a blank report
  48. 48. Enter Metrics & Dimensions from Multi Channel Funnel API Full list here
  49. 49. Hidden secret to making MCF work
  50. 50. Expand hidden rows and enter “mcf” in type
  51. 51. We can run for source, source / medium keywords, campaign etc
  52. 52. The Code
  53. 53. https://gist.github.com/boosterbox
  54. 54. Applications of a better allocation? ● Better Bidding ● More accurate Budget allocation
  55. 55. Testing
  56. 56. Science ● Control & Treatment ● Dependent Variable ● Independant Variable
  57. 57. Science ● Control & Treatment ○ Split target location in smaller locations ○ Split 50/50 ● Dependent Variable ○ Existing attribution model in one half of locations ○ Markov in other half of locations ● Independent Variable ○ Conversions/CPA
  58. 58. Control and Treatment Identify sub-regions DMAs in US
  59. 59. Control and Treatment Split so volume of conversions is roughly even
  60. 60. Dependant Variable Campaign Targeted Locations Attribution model Awesome Campaign 1 Random locations group 1 Last Click Awesome Campaign 1 Test Random locations group 2 Markov Duplicate every campaign
  61. 61. Independent Variable Measure Results

×