Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

1530 track 2 stuart_using our laptop

344 views

Published on

#EMSNYCDAY2

Published in: Marketing
  • Be the first to comment

  • Be the first to like this

1530 track 2 stuart_using our laptop

  1. 1. Is Multi-Touch Attribution Really the Holy Grail of Marketing Measurement? Confidential: Cannot be shared without permission from the Mobile Marketing Association What marketers need to drive marketing productivity in a digital, mobile age An output of MMA’s O c t o b e r 2 0 1 7 Greg Stuart CEO MMA greg@mmaglobal.com
  2. 2. Wow, What a F$^#*@$^ Mess This MTA Thing Is! Confidential: Cannot be shared without permission from the Mobile Marketing Association Some of MMA’s Learnings from a 2-Year Journey to get Good Marketing Productivity into the Marketplace An output of MMA’s O c t o b e r 2 0 1 7 The title they should have used Greg Stuart CEO MMA greg@mmaglobal.com
  3. 3. Today’s Agenda 1. Who is the MMA and what is MATT? 2. What Does MMA’s MATT Know? 3. How is MMA Trying to Help? 3 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  4. 4. Today’s Agenda Confidential: Cannot be shared without permission from the Mobile Marketing Association 1. Who is the MMA and what is MATT? 2. What Does MMA’s MATT Know? 3. How is MMA Trying to Help? 4
  5. 5. MMA is the Industry Trade Assoc. for Mobile Marketing 800+ members WW Operating in 12 countries Started in 2003; turned around in 2013 Marketer led; plus media sellers, tech and agencies 50 staff worldwide 20+ events in 15 countries MMA APAC MMA EMEA MMA LATAM MMA No. Amer.
  6. 6. MMA Global Board of Directors - 65% Marketers Sanjay Gupta Chair CMO Susan Canavari Chief Brand Officer Ilonka Laviz Global Digital Mktg Dir., Global eBusiness Peter McGuinness Chief Mktg & Brand Officer Kellyn Kenny VP, Marketing Tim Mahoney CMO Marc Mathieu CMO Karin Timpone Global Mktg Officer Andrew Sherrard CMO Allan Thygesen President, Americas Jeff Lucas CRO Matt Derella Global CRO Peter Hamilton CEO Tom Kenney CEO John Trimble CRO Will Kassoy CEO Dave Morgan CEO Cameron Clayton CEO & General Manager Suzy Deering CMO Rebecca Messina CMO Ayesha Gilarde CMO Wanda Young CMO Tony Rogers CMO David Godsman CDO Michael Donnelly NA Regional Rep SVP, Global Digital Marketing Ernesto Echeverri LATAM Regional Rep Dir. Mktg USA, Canada & Caribbean Alberto ‘Banano’ Pardo LATAM Regional Rep Founder & CEO Louis Paskalis NA Regional Rep. SVP Enterprise Media Executive Bank Of America David Lowes EMEA Regional Rep CMO Carolyn Everson Global Secretary VP, Global Mktg Solutions Luis Di Como Exec Comm at Large SVP, Global Media Jack Philbin Exec Comm at Large Co-Founder & CEO Stephen McCarthy Global Treasurer CFO (xAd) John Costello Global Chair Emeritus Former Pres., Global Mktg & Innovation Greg Stuart MMA CEO Mobile Marketing Association
  7. 7. 0% 10% 20% 30% 40% 50% 60% Privacy Mobile Messaging Ad Blocking Ad Fraud Internet of Things Mobile Social Mobile Shopper Marketing Mobile Native Viewability Mobile Creative Mobile Data Accuracy Mobile Video Location Targeting Mobile Programmatic Effectiveness and ROI Measurement Measurement and Attribution Marketers Total According to past MMA membership research, Measurement and Attribution tops the needs list N=507, Here is a list of themes that relate to the mobile marketing ecosystem. Which of them are currently more important priorities for your company? 7 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  8. 8. Former Chief Research Officer of the ARF Joel Rubinson Rubinson Partners, Inc.| President Email: joel@rubinsonpartners.com Web: http://blog.joelrubinson.net MMA’s MATT Subject Matter Expert
  9. 9. But What Led MMA Board to Believe MTA was Important In 2013 MMA raised $3 million to Gain an Understanding of the Value/ROI of Marketing Mix in the age of Mobile 9
  10. 10. SMoX Was Our Answer So far We Have Measured 11 In-Market Campaigns 10 Association of National Advertisers American Association of Advertising Agencies Partners 2 QSR
  11. 11. TV Print Internet FSI Cinema Social Radio OOH Mobile Display Display, video, social Display, video, native, location Display, audio, video social Display, video, social Display, video, social Display, video, social Display, RM, Weather targeting, social, video Display, Video, behavioral, retargeting, location, contextual Display, Video, RM, social Contextual, location, daypart11 2014 2015 2015 2015 2016 2016 2016 2016 2017 2017 Each Campaign Measured a Variety of Media Mix QSR
  12. 12. SMoX Methodology: 12 Exposure to Advertising Media Who you are and other context variables (where you are, have you purchased before, time of the day, weather, etc) Outcomes Did your opinion change? Did you download the app? Did you go to the store? Did you buy? Our approach is based on “person level” data. Goal is to isolate and measure the impact of mobile and various other factors in driving “outcomes”
  13. 13. Offline Sales Data From customer Repository of Person Level Data, with PII information for automated matching Anonymized and integrated data at the individual level across all data sources Complete Dataset to Marketing Evolution for analytics Media Exposure Ad Agency/Trading Desk/ DSP Survey Sample Attitudinal metrics & Psychographics $ $ $ $ *Alternate processes are available ig LiveRamp is not an option for the customer specific situation. SMoX Methodology: How data is connected 13
  14. 14. Social Search DMP/CRM Purchases $ $ $ Geo-Location Zip code 5 miles from store Media & Creative Brand Perceptions Very Familiar Likely to Purchase Demographics 23 yrs. old Model Data Set: The data set from LiveRamp includes information on the customer’s demographic profile, media consumption, online behavior, and purchasing history. 1 0% 20% 40% 60% 80% 100% 0 5 10 15 20 ImpactonKPI Frequency of Exposure Frequency to Lift Response Functions Logistic Regression Modeling: Message Impact within a Media: The dataset is modeled to determine the relationship between media and creative exposure and the campaign goal (e.g. sales, awareness, etc.). Interactions: From our logistic regression modeling we also determine how media and creative impact varies by different consumers. 2 TV Ad A Digital Ad 1 Digital Ad 2 TV Ad B $ $ $ Zip code 1 mile from store Zip code 10 miles from store A B 1 A B C 1 2 Not Familiar Not Likely to Purchase Very Familiar Very likely to Purchase 50 yrs. old 45 yrs. old Methodology: Person Level Analysis & Optimization 14 Putting it all together: Marketing Evolution then combines message impact from the logistic regression model, demographic interactions, and media reach, frequency, composition and cost. 3
  15. 15. SMoX Historical Patterns in Optimized Levels of Mobile in the Marketing Mix are.. 15 1% 5% 4% 6% 5% 12% 11% 16% 26% 20% 15% 12% 15% 15% 19% 17% 17% 33% 0% 5% 10% 15% 20% 25% 30% 35% AT&T Gold Peak Tea/Coca Cola MasterCard Walmart 825 Coca Cola China Coca Cola Brazil Unilever Magnum Allstate Top 5 QSR % of mobile display in the mx (ex search) Optimal allocations based on SMoX Studies considering relative ROA/efficiency vs other media Gap to Optimal Actual Spend
  16. 16. And SMoX Historical ROI when Mobile is Optimized in Mix and within Mobile itself (vs. same budget, no mobile) 16 18% 7% 17% 14% 16% 60% 13% 7% 166% 0% 25% 50% 75% 100% 125% 150% 175% AT&T Gold Peak Tea/Coca Cola MasterCard Walmart 825 Coca Cola China Coca Cola Brazil Unilever Magnum Allstate Top 5 QSR % of mobile display in the mx (ex search) Increase in Key Business Outcomes Based Mobile in the Mix vs. No Mobile Increase in Key Business Outcomes Sales SalesSalesSalesSalesBrand Image Sales Store Traffic
  17. 17. 17 It is Crazy that we Think We Can Look at Creative and Know if Is Working/Effective 17 Branding: Aided Awareness/ $ spent* Sales: ROI Mobile Display with weather targeting Mobile Display Campaign Average (across all media) 100 100 *Index is based on Number of people who became aware of Magnum per $ spent. 200 147 175 ZERO
  18. 18. *Index is based on Number of people who became more likely to consider Allstate / $ spent We’ve Consistently Believed Targeting Improves Performance. But How Much is Question! Mobile Video Targeting Consideration / $ Spent + Behavioral targeting + Contextual targeting Demographic targeting 100 18 320 191
  19. 19. New Data Integrations Combined with Personalization of Mobile Creates New Opportunities (for offer oriented QSR campaign) Foot traffic / $ spent* Commuter Coupon User Campaign Average (across all media) 100 518 500 19 Top 5 QSR
  20. 20. …How else do you assess the value of ad unit combination #159: Or, Mobile video channel targeting men who have visited a dealership in the last 7 days …How else do you assess the value of ad unit combination #395: Or, Mobile audio channel targeting women who commute by a QSR regularly over last month …Versus the 500+ other ad format, data, target, time of day, context, location, etc. combinations Based on That, the Case for MTA is…: 20
  21. 21. A Year Ago, We Announced… 21 MATT is a community of industry experts committed to rethinking the world of marketing measurement and attribution; seeking to give marketers better measurements, tools and confidence in connecting marketing to business outcomes. All MMA members are invited to participate in MATT
  22. 22. With a Powerhouse MTA Steering Executive Committee to guide initiative strategy Luis Di Como SVP/Global Media Unilever Lou Paskalis SVP/Enterprise Media Planning, Investment & Measurement Exec. Bank of America Amit Shah CMO 1-800-Flowers.com Sanjay Gupta EVP/Marketing, Innovation & Corporate Relations Allstate Insurance Company 22
  23. 23. And Amazing List of MMA Board Member Participants to support MATT Leadership 23
  24. 24. GlobalBoardKicksOffProject MATT is now Nearly 2-Years in the Learning CMO Review of RFI submissions and 1:1 interviews Provider Evaluation MTA Selection Tool Development RFI to Providers 500 pages of Responses from 19 qualified providers 15 In-depth Discussions + Academics Quantitative Survey I of Marketers N = 190 Input from Joint MTA Board Committees Guidance for Marketers on how to choose the right providers Marketer Needs Assessment MTA Board Task Force Feedback Quantitative Survey II of Marketers N = 400 Technical Analytics Experts Analytics Expert Land- scape Report How-To RFI MTA Scoring Tool Glossary 30 Hours Phone Review 4-Part MTA Webinar Series for Marketers 2016 2nd Half 2017 1st Half MTA Provider Interviews Working Group Formation 24 Data Quality & Accuracy Working Group MTA Success Workbook 2017 2nd Half Color Key: Light Blue = Inputs from Marketers and Vendors Aqua = Inputs from Survey, RFI, or Interviews Dark Blue = Outputs in form of MTA Acceleration Tools Walled Gardens Working Group Input from 78 Member Marketers Data Strategy Guide Data Acquisiti on RFI Marketer Survey III MATT Data Map 2.0 WG Position Paper
  25. 25. Today’s Agenda 1. Who is the MMA and what is MATT? 2. What Does MMA’s MATT Know? 3. How is MMA Trying to Help? 25
  26. 26. The MATT Marketers Defined MTA as… 26 Multi-Touch Attribution: The science of using advanced analytics, • on user level data, • to allocate proportional credit, • across a granular list of marketing touchpoints across many, • and hopefully all, online and offline channels, leading to a desired customer outcome. Excluded: Traditional MMM, brand tracking and last-touch attribution methods Confidential: Cannot be shared without permission from the Mobile Marketing Association
  27. 27. MMA Survey’s: 1/3rd of marketers currently use MTA – And 3/4er’s will be using it in 18 months 34% 34% 49% 66% 76%15% 17% 10% 25% 0% 20% 40% 60% 80% 100% Yes we curently use MTA We will use in 6 months We will use in 12 months We will use in 18 months We dont plan to use in the near future 75% 81% of current users use one of the main MTA providers. The rest uses other providers or internal solutions Does your company currently use a multi touch attribution (MTA) solution or do you plan to use one in the future? N=412, Total Sample 27 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  28. 28. Most marketers use MTA selectively and many are Unsure the Benefits outweigh its Cost. Majority of marketers use MTA selectively, not across all of their budget Six out of 10 marketers think their MTA solution, on average, drives some incremental results or ROI Yet, the majority are not convinced the cost of MTA is worth the benefits 35% 18% 16% 17% 15% Less than 10% 10% to 30% 30% to 50% 50% to 70% 70% to 100% 5% 28% 22% 11% 2% 32% No increase Up to 10% on average Up to 10-20% on average Up to 20-40% on average 40% and above Not sure / Don’t Know 2% 10% 27% 43% 18% Definitely not Probably not Not sure Yes, probably Yes, definitely What share of your total marketing budget (including advertising) do you assess roughly speaking, using a Multi-Touch Attribution solution? (Please consider whether you use MTA for all or just some of your marketing activities and channels) Based on your experience, what do you think is the average impact/lift of using your MTA solution on the total ROI of your marketing activities? Does your current MTA solution pay off for its cost, in terms of driving incremental ROI and impact for your marketing spend to justify its cost? N=107, 28 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  29. 29. The Problem is that the 34% that Utilize MTA Hate it. REALLY, hate it! MMA interviews and a quantitative survey reveal the following marketer views on MTA: -29% -35% -30% -25% -20% -15% -10% -5% 0% MTA providers have a dismal Net Promoter ScoreLow Satisfaction. Marketers not happy with the data they are getting. Fragmented Provider Ecosystem. Top 10 providers account for only two-thirds of use. Mistrust and Hesitation. Marketers have a huge lack of trust, and hesitate to use MTA.** Minimal Expert Understanding. Marketers don’t have MTA expertise. Different Approaches. 19 providers, 25 modeling methods Overall how likely is it you would recommend your main multi touch attribution (MTA) provider? N=118, Total MTA users 1NPS style calculation created from 10 pt satisfaction question 29 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  30. 30. And for Good Reason. We found 25 different modeling techniques - a prescription for chaos! 30 1. Agent-based modeling 2. Bayesian machine learning 3. Bayesian shrinkage 4. Control theory 5. Counter-factuals 6. Doubly robust propensity modeling 7. Elastic net 8. Ensemble based probabilistic 9. Experimental design 10. Frequent pattern analysis 11. GLM 12. Hidden stage Markov models 13. Hierarchical regression 14. LASSO 15. Last touch 16. Logistic regression 17. Monte Carlo simulation 18. Probability of exposure 19. Shapley values 20. Structural equation models 21. Survey based measurement 22. Time decay 23. Time series 24. Utility theory 25. Vector autoregression Confidential: Cannot be shared without permission from the Mobile Marketing Association
  31. 31. And Marketers REALLY want Confidence 31 2% 3% 7% 5% 5% 7% 8% 5% 16% 15% 28% 5% 15% 7% 5% 7% 10% 5% 5% 16% 13% 13% 3% 5% 11% 18% 10% 7% 8% 13% 3% 8% 13% 2% 7% 3% 8% 11% 13% 18% 13% 8% 10% 7% 10% 3% 11% 5% 11% 8% 7% 13% 8% 11% 11% 0% 10% 20% 30% 40% 50% 60% 70% 80% Enhance Mobile Readiness Breaking down “Walled Gardens” Data Quality and Accuracy Standardize and leverage Unified IDs Single source Linkage to sales data Facilitate Agile Marketing Specific approach for Offline Media Transparency of Approach Validation of results and outcomes Use MTA for both Brand and Performance goals Prove lift in Campaign Performance Most Important 2nd 3rd 4th 5th Below is a list of areas that marketers have identified as important priorities to improve the value they receive from MTA solutions. Which of these priorities would make MTA more valuable to you and your organization. Please rank the top 5 with 1 being most important. Marketers need CONFIDENCE Confidential: Cannot be shared without permission from the Mobile Marketing Association
  32. 32. But it Appears There is Not Acceptable Proof of MTA Performance from ANYONE 1. Model predicts “what if” results that are proven true via A/B experiment outcome (“better outcome…”) 2. Relative effectiveness of different touchpoints is proven by experimentation (“got there for the right reasons…”) 3. If analysis is repeated on the same data, you get the same results (“reliable” and transparent, not hocus pocus”) “We measure incremental lift vs. those who received no media exposure”. [Flawed approach] ”we measure the fit of predicted to actual conversion/non-conversions” [flawed: easy to be 99% right by just guessing non-conversions.] “In one case, we…” [anecdotal] Net, not much or any proof! But… 3 characteristics of MTA analytics accuracy we were looking for: What providers offered as “proof” was not satisfying 32 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  33. 33. 1. MTA project starts as a “Big Data” project, but marketers are not data architects But the Real Problem? YOU [marketers] AREN’T READY FOR MTA!!! “MTA is first and foremost a big data project and many big data projects fail” - MTA Provider 33 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  34. 34. 1. MTA project starts as a “Big Data” project, but marketers are not data architects 2. Data quality is not validated 3. Data completeness is needed, therefore walled gardens are a challenge 4. Marketer expectations are higher than what providers can deliver when data assets are not in shape But the Real Problem? YOU [marketers] AREN’T READY FOR MTA!!! “MTA is first and foremost a big data project and many big data projects fail” - MTA Provider “Until we trust the data no one will accept the MTA analysis that is built on it” - Marketer 34 Confidential: Cannot be shared without permission from the Mobile Marketing Association
  35. 35. MMA Identified 12 Elements of “Good MTA” 35 Elements of Attribution Solutions Education Definitions Needed Code of Conduct Best Practices Validation Research Standards Auditing Para- meters Transparency of Approach X X Validation of Results and Outcomes X X X Standardize and Leverage Unified IDs X X X X X Prove Lift in Campaign Performance X X X Specific Approach for Offline Media X X X X X Facilitate Agile Marketing X X Single Source Linkage to Sales Data X X X X Comprehensive Answers for Planning & Budgeting X X X Enhance Mobile Readiness X X X X X X X Use MTA for Both Brand and Performance Goals X X X Data Quality & Accuracy (and Walled Gardens) X X X X X X Experimental Design X X X X
  36. 36. Today’s Agenda 1. Who is the MMA and what is MATT? 2. What Does MMA’s MATT Know? 3. How is MMA Trying to Help? 36
  37. 37. Applied to 34.7% of Campaigns (on avg.) 34% Already Have an MTA Solution MATT’s MTA Mission… This is where we started -29 NPS! 37 Based on MMA Survey from November 2016.
  38. 38. Started with “Selecting…” Launched in 2016 38 * The Decision Guide is available for members only at http://www.mmaglobal.com/matt. The Report: A comprehensive guide to MTA MTA RFI Template Scoring Tool: To help with evaluation 4-Part MATT MTA Webinar Series Part 1: Intro to Multi-Touch Attribution (MTA) Methods Part 2: Selecting the Best MTA Provider For Your Needs Part 3: Making Sense of Attribution Approaches Part 4: Leveraging MTA to Improve Marketing Effectiveness
  39. 39. MMA Released 2 Weeks Ago…. 39 + First two of five new MTA Acceleration Tools to help Marketers apply MTA with Confidence.
  40. 40. North America 1.) MMA MTA Data Map™ 40 One Visual for MTA Strategy Conversations • Poster-sized framework for a more complete and collaborative picture • Used by marketer, agency, DMP, and MTA provider to ensure nothing gets “left out” of the strategy discussion Four Buckets for Thinking about Data 1. Linkable 2. Aggregate 3. Profiling 4. Conversions Confidential: Cannot be shared without permission from the Mobile Marketing Association
  41. 41. Value to Marketers Most researchers, marketers, and analytics people are not data architects, so this guide helps all teams understand the data assets it will take to achieve success with MTA. The MTA Data Strategy Guide… 1. Provides a section by section explanation of what should be considered in the MTA data planning process 2. Defines Data Asset Types and Relational Structures 3. Defines Linking of different types of data 2.) Data Strategy Guide 41 “MTA is first and foremost a big data project and many big data projects fail…” - MTA Provider
  42. 42. More “Applying…” Coming in Late 2017 42 MTA Success Workbook MTA Data Strategy Guide Data Acquisition RFI Position Paper on Walled Gardens Up Next…
  43. 43. Summary…the marketer journey from Point A to Point B (and what the MMA is doing to make change happen) 43 34% adoption -29 NPS Less than 10% impact on marketing ROI Select and Apply with Confidence • MTA selection tools • MTA big data readiness tools • Other validation, white papers, tools 75% adoption Positive NPS 20%+ impact on marketing ROI Join
  44. 44. Thankyou! 44 1-800-Flowers.com* Dunkin Brands* MillerCoors* Colgate-Palmolive MetLife Alex Treglia Philip Solomon Bill Cramblit Danielle Koffer Joseph Presson Stephen McDonagh Ford Motor Co* Pamela Caruth Marilyn Rice Microsoft Allstate* Artur Timotheo Samsung* Shelley Ivanko Kishore Krishna John Baronello Dennis Bulgarelli Jesse Lasakris Coty Inc. Michele Garner Pamela Moy Sharon Russel Marina Koletis Kristina Kaganer Rentola Olli American Express OPEN* General Motors Corp* Stephen Murray Expedia Inc Mondelez International Brian Coleman Chris Hurst T-Mobile* Ameya Karvir Michele Levedag Greg Bongen Joe Mazeika Gavin Olmstead Vinoth Kumar PetSmart Inc Hy Nguyen Laura Hernandez-Romine Mark Roettgering Gap Michelle Mader Ian Mcdonald GlaxoSmithKline* Millie Chu Chris Martinez Pfizer Consumer Healthcare Kayla Cohen Philomena Luk Target Brands, Inc.* Hallmark Tara Thomas Maribeth Crane Scott Reep Meghna Sinha Kemp Strickler Safe Auto Insurance Company Niki Arya Terri Coscia The Coca-Cola Company* Stacy Yehle Sloane Stegen Bank of America* JP Morgan Chase Bank* Gregory Pharo Hotels.com SUBWAY® Abby Mehta Aaron Smolick Leana Less Kevin Roche Amy Bytell Chobani* Chris Hurlebaus Uber* Johnson & Johnson TD Ameritrade Danielle Cherry Marriott International* Kim Larsen Aleks Petkovski Michael Bosco Choice Hotels* Carlisle Connally Travis Smith Igor Levin The Wendy's Company Lindsay Coffelt Stephanie Eilers Vijay Raj Kellogg Co James Bennett Olga Nielsen Mastercard Worldwide* American Family Mutual Ins Co Janelle Bowman USPS Sarah Searls Curt Fournier Andrew Jakubowski McDonalds Corporation Raymond Van Iterson Nilambuj Singh Campbell's Soup Bryan Duffy Sachin Agarwal Kakoli Seal Jennifer Feldman Tim O'Brien Abakus (SAP) Analytic Partners Convertro Neustar / MarketShare Marketing Evolution * = MMA Board Member
  45. 45. Is Multi-Touch Attribution Really the Holy Grail of Marketing Measurement? Confidential: Cannot be shared without permission from the Mobile Marketing Association What marketers need to drive marketing productivity in a digital, mobile age An output of MMA’s O c t o b e r 2 0 1 7 Greg Stuart CEO MMA greg@mmaglobal.com So, to our core question…?
  46. 46. We Don’t Know Yet But we Sure as Heck Are Trying to Figure out The Promise might be bigger than we think but the hurdles are big and audacious. Perfect for a group working together to make the world of marketing better and more modern An output of MMA’s O c t o b e r 2 0 1 7 Want to join our struggle?
  47. 47. Never doubt that a small group of thoughtful, committed people can change the world. Indeed, it is the only thing that ever has. -- Margaret Mead Cultural anthropologist 47 Thank you Greg Stuart CEO MMA greg@mmaglobal.com +1 631 702 0682
  48. 48. North America Using the MMA MTA Data Map™ 48
  49. 49. North America 49
  50. 50. North America 50
  51. 51. 51 Applying

×