Raab Marketing Measurement: Breaking Down Silos for Breakthrough Results

532 views
427 views

Published on

Measuring marketing performance isn't easy, but it's the only way to prove the value of marketing to senior managers and to improve marketing results. This session by consultant David M. Raab shows how to build a complete analytical environment including a formal business model, structured testing process, and integrated technology platform.

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
532
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • what’s the problem? - lots of new channels; each takes an expert; don’t know how to measure, prioritize or compare; don’t have time or money to try them all; on top of usual pressures to produce immediate results, be more nimble; not enough to run each channel separately -- must coordinate across media for best impact - so, the problem is: how do you optimize your marketing mix; i.e., decide which channels to use and how to use them, and implement your decisions
  • short answer: testing - establish objectives for new media; test to see how well can deliver; plan for deployment and execute - recognize that results will improve over time
  • long answer: analytical environment - need to put individual tests into context - requirements for this include: - disciplined process to gather resources, design meaningful tests, read results, AND DEPLOY successes - baseline business model to understand current environment, how existing programs perform, how they interact and relate to long-term results - technology platform to execute tests and, in particular, gather results; although typically execute tests externally, MUST gather consolidated results and must have long-term path to integration
  • long answer: analytical environment - need to put individual tests into context - requirements for this include: - disciplined process to gather resources, design meaningful tests, read results, AND DEPLOY successes - baseline business model to understand current environment, how existing programs perform, how they interact and relate to long-term results - technology platform to execute tests and, in particular, gather results; although typically execute tests externally, MUST gather consolidated results and must have long-term path to integration
  • - document contributors to customer value, e.g. acquisition costs + retention period + value/year; could be more granular if you have the data - then, drill into contributing factors e.g. acquisition cost and customer value by channel, by customer segment; cost and impact of retention programs by channel and segment; drill into funnel stages e.g. awareness, interest, desire, action LTV = Acquisition Cost + (Value/Year x Years/Customer) - ideally, find industry benchmarks to see where you’re weak or strong; can be hard; even without, can set hurdles relative to own performance
  • - document contributors to customer value, e.g. acquisition costs + retention period + value/year; could be more granular if you have the data - then, drill into contributing factors e.g. acquisition cost and customer value by channel, by customer segment; cost and impact of retention programs by channel and segment; drill into funnel stages e.g. awareness, interest, desire, action LTV = Acquisition Cost + (Value/Year x Years/Customer) - ideally, find industry benchmarks to see where you’re weak or strong; can be hard; even without, can set hurdles relative to own performance
  • - define objectives (based on company goals e.g. acquisition, new products, retention, etc., weaknesses in existing funnel ), identify alternatives (based on industry info, understand how they are used, likely results, suitable industries, prerequisites in data, skills, technology ), assess resources (skills, budgets, data); select by matching alternatives, resources & goals to prioritize (recognize different alternatives have different applications); desi gn proper tests (control groups, measurable data, large enough to read), execute , read results (short- and long-term), deploy results; learn and refine (must have standard, visible reporting so can’t bury mistakes; Aberdeen: disciplines process is one of key distinguishing features for best in class); should set aside reasonable budget for testing – fail quickly - marketing mix modeling: formal modeling to optimize allocations; works with known, stable rather than unknown, new media; also need but is refinement applied to stable industry - consider ‘meta process’ of portfolio of tests, balance across channels, anticipated changes in business (e.g. move towards Internet)
  • - define objectives (based on company goals e.g. acquisition, new products, retention, etc., weaknesses in existing funnel ), identify alternatives (based on industry info, understand how they are used, likely results, suitable industries, prerequisites in data, skills, technology ), assess resources (skills, budgets, data); select by matching alternatives, resources & goals to prioritize (recognize different alternatives have different applications); desi gn proper tests (control groups, measurable data, large enough to read), execute , read results (short- and long-term), deploy results; learn and refine (must have standard, visible reporting so can’t bury mistakes; Aberdeen: disciplines process is one of key distinguishing features for best in class); should set aside reasonable budget for testing – fail quickly - marketing mix modeling: formal modeling to optimize allocations; works with known, stable rather than unknown, new media; also need but is refinement applied to stable industry - consider ‘meta process’ of portfolio of tests, balance across channels, anticipated changes in business (e.g. move towards Internet)
  • - ideally, central customer database w/data from all channels linked to individuals - this works in database marketing, where control most messages; not so much in either brand marketing or (some) online marketing, where may not know individual customers at all, or may have semi-anonymous ID e.g. cookie, site registration; often can link via email address but not always; if not, then can still use channel-specific measures with high coverage, or samples against panels or surveys; better approximately right than precisely wrong; increasingly possible to integrate data e.g. social media handle, demographics/clusters based on IP geotargeting, behavioral segments within ad networks - cross channel in particular requires central execution platform: starts with profile, includes scoring, message selection, lists for outbound direct mail / email / phone; realtime interactions by Web, phone; possibly ad serving; social media tracking and execution e.g. case mgt, sentiment analysis, influence measurement; also content mgt, selection & reuse (“content grazing”); integrated testing and tracking
  • - ideally, central customer database w/data from all channels linked to individuals - this works in database marketing, where control most messages; not so much in either brand marketing or (some) online marketing, where may not know individual customers at all, or may have semi-anonymous ID e.g. cookie, site registration; often can link via email address but not always; if not, then can still use channel-specific measures with high coverage, or samples against panels or surveys; better approximately right than precisely wrong; increasingly possible to integrate data e.g. social media handle, demographics/clusters based on IP geotargeting, behavioral segments within ad networks - cross channel in particular requires central execution platform: starts with profile, includes scoring, message selection, lists for outbound direct mail / email / phone; realtime interactions by Web, phone; possibly ad serving; social media tracking and execution e.g. case mgt, sentiment analysis, influence measurement; also content mgt, selection & reuse (“content grazing”); integrated testing and tracking
  • - ideally, central customer database w/data from all channels linked to individuals - this works in database marketing, where control most messages; not so much in either brand marketing or (some) online marketing, where may not know individual customers at all, or may have semi-anonymous ID e.g. cookie, site registration; often can link via email address but not always; if not, then can still use channel-specific measures with high coverage, or samples against panels or surveys; better approximately right than precisely wrong; increasingly possible to integrate data e.g. social media handle, demographics/clusters based on IP geotargeting, behavioral segments within ad networks - cross channel in particular requires central execution platform: starts with profile, includes scoring, message selection, lists for outbound direct mail / email / phone; realtime interactions by Web, phone; possibly ad serving; social media tracking and execution e.g. case mgt, sentiment analysis, influence measurement; also content mgt, selection & reuse (“content grazing”); integrated testing and tracking
  • can’t properly select what to test until you know what your business needs are, per your business model
  • - identify alternatives 5 SM app types (see Silverpop ppt) (25% generate awareness, 18% customer engagement, 16% engage analysts/influencers, 15% market products, 14% monitor and respond, 8% competitive tracking, 4% other); different needs for each e.g. network ads, mobile apps, email names, leadership content, blog traffic; Toyota: monitor & respond; had previous experience e.g. with Scion ‘crowdsourcing’ for customization
  • - define objectives: (depends on situation – awareness, interest, decision, action); Toyota: maintain customer loyalty, address uncertainties
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - select: match alternatives, resources & goals to prioritize (matrix & scoring):
  • - design proper tests (set goals, pick metrics, define size and timeframe, estimate value): metrics - web traffic, qualified leads, customer loyalty/retention, brand awareness, revenue; Toyota: traffic & attitudes
  • - execute : plan carefully; use external resources as necessary; monitor execution closely for glitches; ensure metrics gathering is in place; Toyota: http://toyotaconversations.com/ ; assigned team of 6-8 people to monitor; reached out to advocates
  • - deploy results (deployment process): scaling is issue in manual social media; need steady stream of content; some campaigns have specific shelf-life but then is hard to justify value (depends on goal – e.g., product intro is one-time anyway); long-term goal is usually to build on-going community
  • Raab Marketing Measurement: Breaking Down Silos for Breakthrough Results

    1. 1. Breaking Down Silos for Breakthrough Results David M. Raab Raab Associates Inc. [email_address] www.raabassociatesinc.com
    2. 2. Marketing, 2010
    3. 3. So Many Choices <ul><li>How do I… </li></ul><ul><li>optimize customer experience? </li></ul><ul><li>integrate social media? </li></ul><ul><li>choose the right technology? </li></ul>-80% -60% -40% -20% 0% 20% 40% 60% Change in Marketing Budget Allocation in 2009 According to US B2B Marketers, by Tactic (% of Respondents) 13% 14% 15% 15% 17% 21% 23% 25% 28% 30% 31% 31% 32% 34% 36% 39% 41% 42% 47% 47% TV advertising Sponsorships Outdoor media Print advertising Tradeshows, conferences (in-person) Radio Direct mail Virtual trade shows Online display ads - banners, pop-ups Public relations Other Web 2.0 media Inside sales/telemarketing Executive breakfasts, seminars and events Blogs Discussion forums, social networks or communities E-mail Webinars, Webcasts Online video, podcasts or rich media Company Website Search Marketing Source: MarketingProfs and Forrester Research, 'B-to-B Marketing in 2009: Trends in Strategies and Spending', March 2009 increase decrease
    4. 4. Short Answer: Test Design test Execute test Read results
    5. 5. Why Testing Isn’t Enough <ul><li>individual tests can’t measure long-term results </li></ul><ul><li>optimal experience is based on interaction of multiple programs </li></ul><ul><li>integrated measurement requires integrated technology </li></ul>
    6. 6. Real Answer: Analytical Environment
    7. 7. Analytical Environment: Three Components <ul><li>formal business model </li></ul><ul><li>structured testing process </li></ul><ul><li>integrated technology platform </li></ul>Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine Orders Marketing Service Web Phone Media Web Email Data Integration Marketing Execution Marketing database retain grow acquire
    8. 8. Formal Business Model <ul><li>define customer life cycle (=lifetime value components) </li></ul><ul><li>identify leverage points within each component </li></ul><ul><li>evaluate differences by channel, by customer segment </li></ul><ul><li>assess incremental impact of individual programs </li></ul><ul><li>compare to past performance, industry benchmarks </li></ul>retain grow acquire
    9. 9. Formal Business Model: Self-Assessment Business Model level 1 level 2 level 3 marketing value measure no reporting; capture response capture response rates and cost/response measure lifetime value, understand customer flow, relationships among programs benchmarking no benchmarks compare vs. own previous performance compare vs. industry standards, best-in-class marketing planning track cost and schedule set targets based on past results & program goals measure short- and long-term impacts, actual vs expected marketing optimization none; allocate spend based on history set targets for media mix based on incremental ROI formal modeling for marketing impact and mix optimization
    10. 10. Structured Testing Process <ul><li>careful test selection </li></ul><ul><li>standardized test execution </li></ul><ul><li>routine deployment </li></ul><ul><li>continuous learning </li></ul>Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine
    11. 11. Structured Testing Process: Self-Assessment Testing Process level 1 level 2 level 3 test selection select test programs randomly define program goals by customer lifestage set lifestage targets and select tests to reach; portfolio approach test design informal testing occasional testing, aimed at immediate response formal plans for all tests, with long-term measures and deployment plans deployment use outside vendors for new media hire specialist staff for new media formal training for new media; organize by customer or product
    12. 12. Integrated Technology Platform <ul><li>central marketing database, as integrated as possible </li></ul><ul><li>individual data for known customers </li></ul><ul><li>segment & survey data for mass media, some online </li></ul><ul><li>add new data sources as they appear </li></ul><ul><li>integrate on multiple keys, not just name/address </li></ul>Orders Marketing Service Web Phone Media Web Email Data Integration Marketing Execution Marketing database
    13. 13. Integrated Technology Platform <ul><li>multi-channel execution platform </li></ul><ul><li>coordinated cross-channel treatments </li></ul><ul><li>mix of batch and real time programs </li></ul><ul><li>dynamic content for cross-channel personalization </li></ul>Orders Marketing Service Web Phone Media Web Email Data Integration Marketing Execution Marketing database
    14. 14. Integrated Technology Platform: Self-Assessment Technology Platform level 1 level 2 level 3 data management silos; no central database marketing database w/ CRM and transactions; no online integrated online w//offline execution separate systems for each medium outbound campaigns from central system inbound and outbound from same system, including real time online testing capabilities none simple a/b response tests long-term stream tests
    15. 15. Benefits of Analytical Environment <ul><li>understand current customer experience </li></ul><ul><li>highlight key leverage points </li></ul><ul><li>systematic testing process </li></ul><ul><li>continuous improvement </li></ul><ul><li>platform for adding new channels </li></ul>
    16. 16. Poll: How would you rate your company’s current ability to manage cross-channel campaigns? <ul><li>level 1: every channel is managed separately </li></ul><ul><li>level 2: we compare short-term results across channels, but don’t integrate them operationally </li></ul><ul><li>level 3: we have integrated online and offline channels and measure the long-term impact on customer value of our programs </li></ul>
    17. 17. Self-Assessment level 1 level 2 level 3 Technology data management silos; no central database marketing database w/CRM and transactions; no online integrated online w//offline execution separate systems for each medium outbound campaigns from central system inbound and outbound from same system, including real time online testing capabilities none simple a/b response tests long-term stream tests Business model marketing value measure no reporting; capture response capture response rates and cost/response measure lifetime value, understand customer flow, relationships among programs benchmarking no benchmarks compare vs. own previous performance compare vs. industry standards, best-in-class marketing planning track cost and schedule set targets based on past results & program goals measure short- and long-term impacts; compare vs expected results marketing optimization none; allocate spend based on history set targets for media mix based on incremental ROI formal modeling for marketing impact and mix optimization Testing Process test selection select test programs randomly define program goals by customer lifestage set lifestage targets and select tests to reach; portfolio approach test design informal testing occasional testing, aimed at immediate response formal plans for all tests, with long-term measures and deployment plans deployment use outside vendors for new media hire specialist staff for new media formal training for new media; organize by customer or product
    18. 18. What About Social Media?
    19. 19. 1. Social Media Does It All goals: explain new product awareness of company create relationship awareness of capability credibility as competent vendor quality of service competitive superiority media strengthen relationship advertising & PR x x SEM and display x x SEO and Web site x x x x x social media x x x x x x x x email, mail, phone x x Web events x x x trade shows x x x nurture email x x x x x x in-person events x x x retain grow acquire
    20. 20. 2. Many Social Media <ul><li>Also: </li></ul><ul><li>photo sharing </li></ul><ul><li>reviews/rating </li></ul><ul><li>location check-in </li></ul>
    21. 21. 3. Many Social Media Objectives <ul><li>Also: </li></ul><ul><li>customer support </li></ul><ul><li>market research </li></ul><ul><li>competitor monitoring </li></ul>
    22. 22. Different Social Media for Different Objectives wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media customer support engage customers monitor & respond marketing (lead gen) awareness engage influencers competitor monitoring market research Objectives x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x
    23. 23. … Also Different Resource Needs staff experience funding web analytics media reach public interest technology customer data Resources x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media
    24. 24. Match Media to Objectives and Resources wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media customer support engage customers monitor & respond marketing (lead gen) awareness engage influencers competitor monitoring market research Objectives x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x staff experience funding web analytics media reach public interest technology customer data Resources x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine
    25. 25. Example: Automobile Recall Wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media customer support engage customers monitor & respond marketing (lead gen) awareness engage influencers competitor monitoring market research Objectives x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x staff experience funding web analytics media reach public interest technology customer data Resources x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine
    26. 26. Example: Automobile Recall Wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media customer support engage customers monitor & respond marketing (lead gen) awareness engage influencers competitor monitoring market research Objectives x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x staff experience funding web analytics media reach public interest technology customer data Resources x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine
    27. 27. Example: Automobile Recall Wikis photo sharing location check-in online video podcasting blogging message boards reviews/ratings Twitter social networks Media customer support engage customers monitor & respond marketing (lead gen) awareness engage influencers competitor monitoring market research Objectives x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x staff experience funding web analytics media reach public interest technology customer data Resources x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine
    28. 28. Design Test Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine Design Issues Automobile Recall <ul><li>goals </li></ul>monitor & respond, customer support <ul><li>metrics </li></ul>Web traffic, sentiment, offline opinion polls <ul><li>size & timeframe </li></ul>huge, on-going <ul><li>value </li></ul>brand impact
    29. 29. Execute & Read Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine Execution Issues Automobile Recall <ul><li>process definition </li></ul>evolving <ul><li>training & monitoring </li></ul>experienced staff <ul><li>external resources </li></ul>as needed <ul><li>metrics gathering </li></ul>part of platform, external as needed Evaluation Issues <ul><li>short vs long term </li></ul>short term: volume, sentiment long term: attitudes, purchase intent, purchases <ul><li>often indirect for social media </li></ul>many ‘lurkers’; measure via offline surveys; track opinion leaders <ul><li>link to individuals or segments </li></ul>group registration, geotargeting, referring sites
    30. 30. Deploy & Learn Assess resources Design test Execute test Identify Options Define objectives Deploy winners Select project Read results Learn and refine Deployment Issues Automobile Recall <ul><li>scalability </li></ul>human monitoring and response is expensive <ul><li>content generation </li></ul>user-generated, repurposed press materials <ul><li>shelf-life </li></ul>short-term project; usually want community <ul><li>data integration </li></ul>use standard systems; low priority <ul><li>knowledge transfer </li></ul>use in-house staff from start Learning Issues <ul><li>consistent reporting </li></ul>ad hoc project <ul><li>insights </li></ul>segment results, use of different content & media <ul><li>adequate budget </li></ul>for routine testing <ul><li>formal optimization </li></ul>best for stable situations <ul><li>portfolio approach </li></ul>balance risk/reward, channel mix, long-term strategy
    31. 31. Lessons Learned <ul><li>analytical environment combines business model, testing and technology platform </li></ul><ul><ul><li>platform is the foundation </li></ul></ul><ul><ul><li>business model is the framework </li></ul></ul><ul><ul><li>testing fills in the details </li></ul></ul><ul><li>all are needed to optimize customer experience </li></ul>
    32. 32. Lessons Learned <ul><li>structured testing process </li></ul><ul><ul><li>match projects to applications and resources </li></ul></ul><ul><ul><li>careful test design: goals, metrics, evaluation </li></ul></ul><ul><ul><li>use outside resources as necessary </li></ul></ul>
    33. 33. Lessons Learned <ul><li>social media is difficult, but not impossible </li></ul><ul><ul><li>many different kinds of social media; must pick carefully </li></ul></ul><ul><ul><li>measurement and data integration are especially challenging </li></ul></ul><ul><ul><li>structured testing is still the key </li></ul></ul>
    34. 34. What Now? <ul><li>Don’t Panic </li></ul><ul><ul><li>new media are a fraction of total spend </li></ul></ul><ul><ul><li>everybody else is learning too </li></ul></ul><ul><li>Make a Plan </li></ul><ul><ul><li>assess platform, business model, and test capabilities </li></ul></ul><ul><ul><li>set long-term objectives </li></ul></ul><ul><li>Execute </li></ul><ul><ul><li>short-term tests for immediate results </li></ul></ul><ul><ul><li>long-term changes to build infrastructure </li></ul></ul>
    35. 35. Thank You email: [email_address] Web: www.raabguide.com blog: http://customerexperiencematrix.blogspot.com/ Twitter: @draab phone: 914-241-2117

    ×