Attribution-fu: Using Correlation Data to Track Marketing Attribution

12,432 views

Published on

Marketing has become extremely data-driven. I don't love it, but we need to go with it. That means finding more and better ways to track attribution across channels. This slide deck explains some basic techniques - Pearson Correlation and holdout testing - that you can use to connect channel performance to business KPIs.

Published in: Marketing, Business, Technology

Attribution-fu: Using Correlation Data to Track Marketing Attribution

  1. BECAUSE MATH DIPPING YOUR TOE IN THE WATER W/ CORRELATION Ian Lurie @portentint ian@portent.com #PORTENTU
  2. portent.com portent.co/bccorrel
  3. HOW THIS WILL GO Disclaimers Use cases & despair Principles 2 ways to test We all collapse
  4. THE LINK BUNDLE portent.co/bccorrel
  5. DON’T BELIEVE A WORD I SAY
  6. MY MATH QUALIFICATIONS A degree in history A JD C- in Calculus No statistics training at all
  7. I’VE MADE MISTAKES
  8. LUCKY FOR YOU, I’VE HAD SOME GREAT TEACHERS
  9. aNNIE CUSHING kevin hillstrom pete meyers avinash kaushik john caples
  10. WHAT I WON’T DISCUSS learn the rules before you use the tools GOOGLE ANALYTICS R LANGUAGE TABLEAU SOFTWARE OTHER FANCY STUFF LOVE IT ALL. BUT let's STICK TO TOOLS WE ALL KNOW AND HAVE, NO MATTER what
  11. HOW DO WE KNOW IF THIS IS WORKING? WE CAN'T TRACK IT. THOSE ARE CALL CENTER LEADS. INTERNET MARKETING HAD NOTHING TO DO WITH IT. WE’VE ALL KNOWN DESPAIR I WANT TO STOP THE CONTENT CAMPAIGN. IT'S NOT EARNING ANY MONEY. I'M AFRAID THE WEB IS JUST CANNIBALIZING DIRECT MAIL. DON'T BUY PPC FOR OUR NAME. WE ALREADY RANK #1. IT'S A WASTE OF SOCIAL MEDIA MONEY. DOESN'T CONVERT.
  12. WE’VE ALL KNOWN DESPAIR gaaaah i can't take it. attribution?!!!! hahahahahah i'll attribute my sanity to finding a job at a bicycle shop bwahahahahahahaha
  13. CORRELATION IS YOUR FRIEND Car Speed Car top speed vs. age of American Male 180 160 140 120 100 80 60 40 20 0 CLEAR CORRELATION 30 35 40 45 50 Age 55 60 65 70
  14. CORRELATION IS YOUR FRIEND WORST ENEMY Car Speed Car top speed vs. age of American Male 180 160 140 120 100 80 60 40 20 0 BUT… WHY? 30 35 40 45 50 Age 55 60 65 70
  15. CORRELATION IS YOUR FRIEND WORST ENEMY MIDDLE-AGED AMERICAN MEN ARE COMPENSATING FOR SOMETHING OR ALL AMERICAN MEN LIKE TOYS, AND CAN AFFORD THEM AT MIDDLE AGE
  16. LATE-NIGHT SNACKING MAKES YOU FAT WRONG Late night snacks vs. weight gain 50 Weight gain (lbs) 40 30 20 10 0 0 2 4 6 Snacks/week 8 10
  17. WHY IS IT WRONG? THERE IS NO CAUSAL LINK. OR, AS STATISTICIANS LIKE TO SAY…
  18. correlation does not equal causation ...but i wouldn't worry about that too much.
  19. (WE LOVE YOU, MATT)
  20. WE CAN MAKE THIS WORK BY USING EXCEL/GDOCS USING OUR KNOWLEDGE GETTING A BIT GEEKY TESTING ASSUMPTIONS
  21. EXAMPLE 1 our e-mail list? make it a low priority. it doesn't sell much anyway.
  22. A HOLDOUT TEST!!! TEST ONE CHANNEL’S IMPACT ON OTHERS
  23. A HOLDOUT TEST!!! SEGMENT YOUR AUDIENCE 5%/5%/90%
  24. A HOLDOUT TEST!!! 5% GETS THE USUAL
  25. A HOLDOUT TEST!!! 5% GETS NOTHING FROM THE CHANNEL BEING TESTED
  26. A HOLDOUT TEST!!! THE RESULT GIVES YOU INSIGHT INTO CROSSCHANNEL IMPACT
  27. OVERALL RESULT (all channels) Average revenue value generated from all channels of a single address List Size Total sales Revenue/address Total 400,000 $250,000.00 $0.63 Held out 20,000 $5,000.00 $0.25 E-mailed 20,000 $12,000.00 $0.60 Revenue from all channels for each segment
  28. OVERALL RESULT (all channels) E-mail caused these customers to buy more, no matter where they came from. Organic Search PPC Social E-mail Total $0.20 $1.25 $0.11 $0.63 Held out $0.17 $0.90 $0.02 $0.25 E-mailed $0.21 $1.24 $0.13 $0.60 Ah HA! Held out segment generated less value from other channels, too.
  29. OVERALL RESULT (all channels) Now you estimate impact by channel. Organic Search PPC Social E-mail Total $0.20 $1.25 $0.11 $0.63 Held out $0.17 $0.90 $0.02 $0.25 E-mailed $0.21 $1.24 $0.13 $0.60 E-mail appears to boost PPC revenue/customer 39%
  30. EXAMPLE 2 SOCIAL MEDIA SUCKS. LET'S STOP. FOREVER.
  31. UH-OH THIS IS ABOUT EXISTING & NEW CUSTOMER ACQUISITION.
  32. UH-OH HOW DO WE TRACK LIFT IN NEW CUSTOMER ACQUISITION?!
  33. OPTION 1 TURN OFF ONE CHANNEL. MEASURE CHANGE IN THE OTHERS.
  34. GENERALLY UNPOPULAR SHIIIIIIIIIIIII
  35. OPTION 2 BOOST SPEND ON ONE CHANNEL. MEASURE IMPACT ON OTHERS.
  36. ALSO UNPOPULAR PLUS, LOTS OF NOISE. YOU PROBABLY CAN’T BOOST SPEND 300%. SHIIIIIIIIIIIII
  37. OPTION 3 IAN’S SEAT-OF-THE-PANTS CORRELATION METHOD
  38. USE AT YOUR OWN RISK. MAY CAUSE
  39. WHAT YOU NEED A LOT OF DATA CLEAN DATA ANALYTICS THAT WORK INTERNAL SALES DATA (NOT JUST WEB) AN OPEN MIND
  40. WHAT YOU NEED COMMON SENSE
  41. STEP 1: GET YOUR DATA YOU MUST HAVE OVERALL SALES!!!!
  42. STEP 2: IMPORT IT
  43. STEP 3: CREATE SCATTERPLOT THIS TESTS WHETHER YOU HAVE A CHANCE
  44. SELECT 2 COLUMNS CMD+SHIFT OR CTRL+SHIFT
  45. WHICH COLUMNS? HERE, WE’RE CHECKING FOR CONNECTIONS BETWEEN SOCIAL MEDIA ACTIVITY AND REVENUE, SO I’M STARTING WITH ONE SOCIAL MEDIA METRIC AND OVERALL REVENUE
  46. CLICK
  47. CHECK THIS BOX
  48. IN GENERAL STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION R SQUARED CLOSER TO 1 =TIGHTER CORRELATION
  49. BUT REMEMBER… I WAS A HISTORY MAJOR.
  50. HMMMM Overall Rev 30,000.00 Revenue (USD) 25,000.00 20,000.00 R² = 0.40636 15,000.00 LOW R SQUARED= LOW 10,000.00 CORRELATION = LOWHMMM. MIGHT BE A CHANCES THESE ARECONNECTION 5,000.00 CONNECTED - 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Visits from Social Media
  51. HMMMM Revenue (USD) PPC Rev $20,000.00 $18,000.00 $16,000.00 $14,000.00 $12,000.00 R² = 0.81774 $10,000.00 $8,000.00 LOW R SQUARED= LOW CORRELATION = LOWHMMM. MIGHT BE A $6,000.00 CHANCES THESE ARECONNECTION $4,000.00 CONNECTED $2,000.00 $1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 Visits from Social Media
  52. HMMMM Overall Rev 30,000.00 R² = 0.30871 REVENUE 25,000.00 20,000.00 15,000.00 10,000.00 WEAKER RELATIONSHIP BETWEEN OVERALL REVENUE AND SHARES 5,000.00 - 20 40 60 80 SHARES 100 120 140
  53. HMMMM REEVNUE (USD) E-mail Rev vs. Social Unique Visits $10,000.00 $9,000.00 $8,000.00 $7,000.00 $6,000.00 $5,000.00 $4,000.00 $3,000.00 $2,000.00 $1,000.00 $- SOCIAL MAY CANNABILIZE E-MAIL R² = 0.04969 - 500 1,000 1,500 SOCIAL UNIQUE VISITS 2,000 2,500
  54. STEP 4: USE CORRELATION NUMBERS, TO QUANTIFY THE PLOTS
  55. TYPE THIS FORMULA INTO A CELL: =CORREL(RANGE, RANGE2)
  56. IN MY EXAMPLE: =CORREL(D2:D363, K2:K363)
  57. CLOSER TO 1 = STRONG CORRELATION
  58. MY RESULT:
  59. MY RESULT: DANG
  60. MY RESULT: HOLY @)(*!@#
  61. correlation does not equal causation
  62. HMMM. A GOOD POINT.
  63. STEP 5: APPLY COMMON SENSE REMEMBER OUR SCATTERPLOT OF SOCIAL SHARES VS. OVERALL REVENUE?
  64. IT’S A PRETTY GOOD ‘FIT’ Overall Rev 30,000.00 R² = 0.30871 REVENUE 25,000.00 20,000.00 REMEMBER OUR 15,000.00 SCATTERPLOT OF 10,000.00 SOCIAL UNIQUES VS. 5,000.00 OVERALL REVENUE? - 20 40 60 80 SHARES 100 120 140
  65. NOT THE STRONGEST, BUT SOMETHING’S GOING ON THERE.
  66. (NO, THE NUMBERS DON’T MATCH UP. JUST SAMPLE DATA. PLUS: HISTORY MAJOR)
  67. MORE COMMON SENSE KNOWING WHAT I’VE DONE IN SOCIAL MEDIA RECENTLY HELPS, TOO. WE HAVEN’T CHANGED A THING. BUT STILL, THIS CORRELATION.
  68. MORE COMMON SENSE WE RAN A PROMO, THOUGH, VIA E-MAIL. THAT MIGHT CREATE ‘NOISE.’
  69. MORE COMMON SENSE SO WE DO THE MATH
  70. MORE COMMON SENSE THAT MAY MEAN THE E-MAIL PROMO LOOSENED THE RELATIONSHIP BETWEEN EMAIL AND SOCIAL. SO WE RUN ANOTHER CORRELATION EXCLUDING THE DAYS OF THE PROMO.
  71. MORE COMMON SENSE THERE’S PROBABLY CANNIBALIZATION GOING ON.
  72. THE RESULT YOU CAN AT LEAST DEMONSTRATE THERE’S A STRONG CONNECTION BETWEEN REVENUE AND SOCIAL, EVEN IF SOCIAL DOESN’T DIRECTLY GENERATE THAT REVENUE.
  73. NEXT STEP? DO A HOLDOUT TEST (COUGH) – STOP POSTING FOR 2 WEEKS. TRY TO REALLY SPIKE SOCIAL MEDIA ACTIVITY AND SEE HOW THAT IMPACTS OVERALL REVENUE.
  74. CAUTION DON’T TRY TO ‘FIT’ THE DATA TO YOUR EXPECTATIONS IF COMMON SENSE <> THE DATA, GO WITH COMMON SENSE UNTIL PROVEN OTHERWISE
  75. CAUTION REALLY LEARN THIS STUFF
  76. QUESTIONS? IAN@PORTENT.COM @PORTENTINT WWW.PORTENT.COM portent.co/bccorrel
  77. NEXT MONTH MICHAEL WEIGAND NEXT-LEVEL SEGMENTATION DIVIDE & CONQUER FEBRUARY 27 TH 11 AM PACIFIC

×