Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution


Published on

This slide presentation is from a webcast sponsored by ClearSaleing and Search Marketing Now. The webcast focuses on using advanced statistical models to build attribution models that can then be tested.

The webinar is co-hosted by Adam Goldberg, ClearSaleing co-founder and Chief Innovation Officer, and Dr. Purush Papatla, President of Vetra Analytics, which is a high-end statistical consultancy group that is partnered with ClearSaleing.

Dr. Papatla shows how using high-end statistical models can allow one to build attribution models that specifically address how to account for:

* Social media
* Word of Mouth
* The differences between short Purchase Paths and long Purchase Paths

As in previous webinars, there are some interactive elements. To view the webinar in its entirety, please visit or:

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution

  1. 1. Attribution Management Forum 3.0: How To Build Accurate Models To Solve Attribution Tuesday May 5, 2009 1 PM EDT Speakers: Adam Goldberg, Dr. PurushPapatla ©2009 Third Door Media, Inc.
  2. 2. Viewing Tips  Turn Off Pop-Up Blockers  Technical difficulties? Click on “Help?” link  Use Q+A box   Submitting questions to speaker Q+A session at end of webcast  Use “Ask a Question” box to submit  questions Send questions at any time  ©2009 Third Door Media, Inc.
  3. 3. Adam Goldberg, Chief Innovation Officer  Co-Founded ClearSaleing Inc. in 2006 Columbus, OH  Started Google’s inside sales organization in 2003-2006 New York City, NY  Started Actuate’s inside sales organization in 2000-2003 San Francisco, CA  Worked for Oracle Corp. in 1998-2000 in Major Account Sales Redwood Shores, CA  Speaker and Trainer at events such as: (SMX), (SES), (DMA) ©2009 Third Door Media, Inc.
  4. 4. Dr. PurushPapatla  Ph.D. from Kellogg School of Management at Northwestern University  Associate Professor, Marketing Sheldon B. Lubar School of Business  President and Founder; Vetra Analytics  Published in top-tier marketing journals » Marketing Science » Journal of Marketing Research » Journal of Business Research » Journal of Retailing » Journal of Interactive Marketing ©2009 Third Door Media, Inc.
  5. 5. EVOLUTION OF ONLINE ADVERTISING Number of Online Offline Attribution Portfolio Clicks Conversions Conversions Management Management ©2009 Third Door Media, Inc.
  6. 6. ATTRIBUTION MANAGEMENT HIERARCHY ©2009 Third Door Media, Inc.
  7. 7. THE CONSUMER BUYING CYCLE ©2009 Third Door Media, Inc.
  8. 8. THE CONSUMER BUYING CYCLE ©2009 Third Door Media, Inc.
  9. 9. THE CONSUMER BUYING CYCLE ©2009 Third Door Media, Inc.
  10. 10. Poll Question 1 • Currently, how are you attributing conversion credit to your various ad sources? 1. Last click 2. Other attribution method ©2009 Third Door Media, Inc.
  11. 11. Recap of Modeling Framework From “Measuring the Immeasurable” ©2009 Third Door Media, Inc.
  12. 12. CONSUMER DECISIONS ©2009 Third Door Media, Inc.
  13. 13. DECISION INFLUENCER What we know What we don’t know yet Our Communications Communications ┼Competitor Search search ┼Paid ┼Consumer U ┼Banner Ads Site visits to competitors ┼ n Product trials ┼e-mail ┼ c Promotions ……. ┼Onsite ┼ ┼Comparison Shopping e ┼Affiliate ad r sources ┼Other Social Media t ┼ Consumer Search Word of mouth ┼ a ┼Organic search Opinion sites ┼ i ┼Site visits to us Expert opinions ┼ n Traditional Mass Media ┼ t y ©2009 Third Door Media, Inc.
  14. 14. MODELING CONSUMER DECISIONS Build a mathematical model to predict consumer decisions ┼ Using data on influencers that we are able to track and measure ┼ Representing data on influencers that we can’t yet track and measure - our ┼ uncertainty - through a statistical distribution Calibrate the model on observed consumer decisions ┼ Purchase - yes/no ┼ Purchase size - dollar volume, # of units ┼ Repeat purchases ┼ Word of mouth ┼ Etc. ┼ Test the model’s quality by comparing predicted and actual behavior ┼ ©2009 Third Door Media, Inc.
  15. 15. CONSUMER DECISION MODEL Consumer’s Decision = f (Our Communications, Consumer Search, Competitor Communications, Other Sources) = f ([Paid Search, Banner Ads, e-mail, Onsite Promotions, Comparison Shopping, Affiliate ads], [Organic search, Site visits to us], [uncertainty]) ©2009 Third Door Media, Inc.
  16. 16. MEASURING THE EFFECTS OF KNOWN FACTORS? We assume that each of the known influencers has an influence potential ©2009 Third Door Media, Inc.
  17. 17. MATHEMATICAL MODEL FOR CONSUMER’S DECISION * The β’s are the attribution weights ©2009 Third Door Media, Inc.
  18. 18. GETTING THE ATTRIBUTIONS We calibrate the model on data from the ClearSaleingplatform The data includes but is not limited to: Purchase Path™ data Record of consumer’s decisions • Purchase/non-purchase •Product(s) purchased • Amount spent • Repeat visits and purchases ©2009 Third Door Media, Inc.
  19. 19. GETTING THE ATTRIBUTIONS Calibrate the model on the ClearSaleing data • Find the values of β’s which will help us predict consumer decisions as accurately as possible Model is calibrated using: •Maximum Likelihood •Bayesian methods Theβ’s are the attribution weights! ©2009 Third Door Media, Inc.
  20. 20. MODELING THE INFLUENCE POTENTIAL Influence potentialof an influencer = f (# of exposures, when each of the exposures occurred, decay rate of the effect of exposures) ©2009 Third Door Media, Inc.
  21. 21. Poll Question 2 • What challenges have you run into when trying to build an attribution model? 1. Our technology cannot track beyond the last ad clicked 2. We cannot build a sound mathematical model 3. We cannot incorporate offline, social media, and word of mouth advertising 4. All of the above 5. We haven’t tried to build an attribution model ©2009 Third Door Media, Inc.
  22. 22. PROGRESS SINCE LAST WEBINAR 1. Selection of businesses for the first round of model testing 2. Identification of unique influencers 3. Set up the data for calibrating and testing the model 4. Calibrate and test multiple versions of the model ©2009 Third Door Media, Inc.
  23. 23. SELECTING BUSINESSES  Selecting businesses that: •Have a high level of ad spend •Wide array of advertising sources (paid search, email, banner, etc) •We have a least 6 month of data We have 2+ years of data in some cases •Seasonal variations ©2009 Third Door Media, Inc.
  24. 24. SELECTED BUSINESS VERTICALS We will be developing and testing the model on nine businesses in the following verticals • Retail – web only • Retail – multi-channel • Insurance • Financial Services ©2009 Third Door Media, Inc.
  25. 25. PROGRESS SINCE LAST WEBINAR Identification of unique influencers ©2009 Third Door Media, Inc.
  26. 26. INFLUENCER CATEGORIES  We organized the influencers into the following categories: •Direct • Organic Referrers (e.g., Google) • Paid Search • Comparison Shopping •e-mail • Display advertising • Affiliate • Social Media • Video ©2009 Third Door Media, Inc.
  27. 27. UNIQUE INFLUENCERS Each category was further sub-categorized into a number of unique influencers: •Direct - 1 • Organic Referrers – 11 (Google, Yahoo, MSN, etc) • Paid Search Engine – 11 (Ex: Brand vs. Non-Brand) • Comparison Shopping – 3 (Ex: Model Number vs. Product Name) • e-mail - 3 (Ex: Direct Response vs. Brand) • Display advertising - 4 • Affiliate - 2 • Social Media - 1 • Video - 1 ©2009 Third Door Media, Inc.
  28. 28. OVERALL We have: • 9 categories of influencers •37 types of unique influencers across the nine categories  Our model develops attributions for these 37 unique influencers across the nine businesses. ©2009 Third Door Media, Inc.
  29. 29. PROGRESS SINCE LAST WEBINAR Set up the data for calibrating and testing the model ©2009 Third Door Media, Inc.
  30. 30. PROGRESS SINCE LAST WEBINAR 135 predictors 64,653 Purchase Paths™ •11,353 paths resulting in a purchase •53,300 abandoned paths that did not end in a purchase oA path was defined as abandoned based on some proprietary criteria  Model can explain abandonment too  Another frontier: Attributions for abandonment ©2009 Third Door Media, Inc.
  31. 31. RESULTS TO DATE  To date, we have calibrated over 70 versions of the model  We plan to calibrate and test the model at least 500 more times in various forms before firming up our conclusions •45,000 models run across the nine data sets  Testing • Do the estimated attributions make intuitive sense? • Is the model able to predict consumer behavior? o Can it predict purchases? o Can it predict non-purchases? ©2009 Third Door Media, Inc.
  32. 32. RESULTS TO DATE  Intuitive assessment of attributions • Findings: not yet firmed up since we have 37 unique influencers to assess across hundreds of model runs  Predictive Testing • 85% or more of the purchases being predicted correctly • 95% or more of the non-purchases predicted correctly • Lift charts for calibration and prediction samples o Top decile indices average between 450 and 500 ©2009 Third Door Media, Inc.
  33. 33. TYPES OF NON-CLICK/ PASSIVE INFLUENCERS  Consumer ratings and reviews  Social networks  Blogging  Social commerce  Instant messaging  Twitter  You Tube  RSS and multiple feeds ©2009 Third Door Media, Inc.
  34. 34. ATTRIBUTIONS FOR NON-CLICK INFLUENCERS  Why do we need to? • Benefits of including non-click influencers in attribution models • Risks of not-including non-click influencers in attribution models  Challenges • How do we include them, if they don’t click and we don’t track?? ©2009 Third Door Media, Inc.
  35. 35. VETRA PASSIVE SURVEY™  Use survey data on the use of non-click influencers • Statistically infer the likelihood of the use of each type of passive influencers by different demographic, lifestyle and psychographic segments •Vetra is currently working on this approach oVetra Passive Survey™ •Vetra Passive Survey™ can be used to include passive influencers in attribution models ©2009 Third Door Media, Inc.
  36. 36. VETRA PASSIVE PROPORTIONS™ Monitor sources of passive influence • YouTube •Facebook •Myspace • Statistically infer the proportions of buyers who engage in discussions and exchanges regarding products •Vetra has completed preliminary work on a model for this inference Vetra Passive Proportions™ can also be used to include passive influencers in attribution models ©2009 Third Door Media, Inc.
  37. 37. ONGOING RESEARCH ON ATTRIBUTION AND NEXT STEPS Vetra and ClearSaleingwill: •Continue to test a number of attribution models and influencers • Analyze the performance of models across different verticals • Identify the best attribution models for different verticals • Expand attribution models to include passive influencers using oVetra Passive Survey™ oVetra Passive Proportions™ ©2009 Third Door Media, Inc.
  38. 38. Poll Question 3 • What is your timeframe in switching from a last click model to an advanced attribution model? 1. Less than 6 months 2. Within a year 3. Within 2 years 4. More than 2 years 5. No timeframe ©2009 Third Door Media, Inc.
  39. 39. ©2009 Third Door Media, Inc.
  40. 40. QUESTIONS? Adam Goldberg- Upcoming SMN webcast:May12: Ask iProspect: Strategies &Tactics for World-Class SEO May 14: Search Marketing for Small Business: The Basics for Online Success ©2009 Third Door Media, Inc.