Online Test, Target and Measurement - Nancy Shaver, Experian Marketing Services


Published on

Online Test, Target and Measurement

The pressure to deliver results and show ROI for all marketing spend is further complicated by market pace changes and economic climate. Learn how to find the optimal mix of ideas, and do it faster to increase ROI on marketing spend and provide relevant content and interactions to target audiences through the different channels including web, email, and mail.

* Nancy Shaver, Consulting Principal, Experian Marketing Services

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Online Test, Target and Measurement - Nancy Shaver, Experian Marketing Services

  1. 1. Online Marketing Summit<br />Washington DC, July 2010<br /> a world of insight<br />Online Test, Target and Measurement<br />Nancy Shaver <br />Consulting Principal<br />Marketing AnalyticsExperian Marketing Services <br />1<br />
  2. 2. Bio<br />Nancy Shaver is a Consulting Principal for Experian Marketing Solutions’ Marketing Analytics team. In this role, she leverages both her strategic vision and extensive hands-on direct, digital and database marketing experience to help Experian’s clients develop and implement achievable marketing solutions that deliver measurable business value. Through a combination of her work across a range of clients including retail, telecommunications, consumer packaged goods, and financial services, as well as her deep operational and strategy experience, Nancy is able to help clients translate their goals into strategies and practical and effective initiatives which link a positive customer experience with bottom line results.<br />Nancy has over 25 years of experience leading marketing organizations as well as delivering data, analytical, and technical services. Prior to joining Experian, Nancy was a principal and founder of 6C Consulting, specializing in optimizing multichannel customer marketing effectiveness. She has also held senior management positions at MCI and Claritas. Other experience includes participation in a campaign management software start-up, as well as extensive account and operations management at a marketing service provider. <br />Nancy has been a speaker at industry conferences and has been a board member of the DMAW Educational Foundation and TransAmerica Marketing. Client relationships have included several of Experian’s large strategic accounts.<br />Nancy holds a Bachelor of Arts from Wellesley College and attended the MBA program at Mary Washington University.<br />
  3. 3. <ul><li>Market Situation
  4. 4. Challenges… and opportunities
  5. 5. Its all in the approach
  6. 6. Drinking the Koolaid</li></ul>Session agenda<br />
  7. 7. Experian overview<br />Marketing Services<br />Credit Services<br />Interactive<br />
  8. 8. Part of $3.8 billion Experian<br />Experian Marketing Services moves marketing forward with predictive insights that improve how marketers target and engage their customers across both traditional and digital media<br />Marketing Services<br />Credit Services<br />Interactive<br />Decision Analytics<br />Experian Marketing Services<br />
  9. 9. Unparalleled data assets that give a 360º customer view<br />Strategic, data-driven expertise to optimize and grow programs<br />Technology to target, segment and engage across channels<br /><ul><li>Industry's largest supply of consumer data
  10. 10. Data includes:Demographics, attitudes, lifestyles, media habits, purchases, auto and credit scores, internet usage, life events, etc.
  11. 11. Rich history as secure, trusted data provider
  12. 12. Over 30 years of proven analytical expertise
  13. 13. Analysis to understand past marketing activities
  14. 14. Predictive models toidentify future success
  15. 15. Establish business strategies to optimize programs and ensure continued growth
  16. 16. Best-in-breed email marketing for enterprise
  17. 17. Cutting edge real-time consumer profiling on web, mobile, TV
  18. 18. True multi-channel data integration
  19. 19. Industry leading address verification & datahygiene tools</li></ul>Core expertise<br />Data<br />Analytics<br />Platforms<br />Experian Marketing Services<br />
  20. 20. Q1 – Why did the French team crash out? <br />Q2 – Why did the Italian team crash out? <br />Q3 – Why did the England team crash out? <br />Q3 – Why did Spain win the World Cup? <br />Quick survey<br />World Cup<br />
  21. 21. Demand for measurable returns have accelerated<br />Channel proliferation continues<br />Elevated consumer expectations<br />Program performance measures are complex and challenging<br />Historical marketing “silos” make integration difficult<br />Where we are<br />New Marketing Ecosystem<br />
  22. 22. Trends in the marketplace<br />CMO Council’s latest State of Marketing report<br />
  23. 23. Trends in the marketplace<br />CMO Council’s latest State of Marketing report<br />
  24. 24. Digital landscape<br />
  25. 25. External<br />Internal<br />Marketers challenges<br />External factors<br /><ul><li>Media breadth and influence continually shifting
  26. 26. Pervasiveness of technology </li></ul>Shifting customer dynamics<br /><ul><li>Demographic and lifestyle changes
  27. 27. Increase in the level of choice within customer viewpoint</li></ul>Marketplace changes<br /><ul><li>Current climate altering consumer behavior
  28. 28. Rapidly changing marketing conditions
  29. 29. Move towards global markets increasing</li></ul>Data growth<br /><ul><li>More information sources – external and internal fragmentation
  30. 30. Discipline and time to absorb in granular nature difficult</li></ul>Internal obstacles<br /><ul><li>Staffs are over-worked, limited budget for add’l FTE
  31. 31. Building a COE not realistically feasible within timeframes</li></ul>ROI based decision making<br /><ul><li>Shift to accountable marketing and scrutiny
  32. 32. Expectations of doing move with limited budgets</li></li></ul><li>Market place situation<br />Current needs – a mixture of old and new<br />How do I segment and target my prospects in the new media environment?<br />Can’t we use purchase history to tell me what I should be offering each customer? <br />We have to cut circulation budgets. How do we do that while preserving revenue?<br />How do we correctly attribute sales to direct mail, email, web and media?<br />We’ve churned through a lot of consumers. How do I get the most value out of my formers?<br />How do I optimize contacts across channels - direct mail, email and web? <br />Reducerisk<br />We’ve optimized response of each campaign. How do I optimize the value of the customer to us?<br />I’m losing as many customers as I acquire every year. What can I do?<br />
  33. 33. Market place situation requires a shift<br />Defensive posture<br /><ul><li>A focus on customer programs
  34. 34. Reduced scope, scale and spend
  35. 35. Optimization, a common core theme</li></ul>Change continues<br /><ul><li>Competitive position and industry players
  36. 36. Consumer attitudes, behaviors and media consumption
  37. 37. Customer aging
  38. 38. Digital technology and media advances</li></ul>Analytic refresh and upgrade cycle<br /><ul><li>Segmentation systems and profiles reflecting who customers were, not who they are
  39. 39. Out-of-date acquisition strategies and prospect models
  40. 40. Customer targeting systems that do not reflect changed customer expectations</li></ul>Shifting to pro-active position has become more complex than just having a plan – it is imperative understand and assimilate all information and channels available<br />
  41. 41. What you need to know…<br />…to plan to address their needs:<br />What each customer buys?<br />Who exactly are my customers?<br />Where are they outside of your brand experience?<br />Customer Lifetime Value<br />Where each customer is all the time?<br />Their cross-channel usage<br />Are they your top brand advocates?<br />Two-way communication with the customer<br />Top of mind competitors<br />Click-stream information<br />What is the right marketing investment for them?<br />Deep segmentation and understanding<br />Customer Satisfaction and Loyalty<br />Why do they buy from you, and how do they buy from you?<br />
  42. 42. Applies to all channels<br />Right audience<br />Customer intimacy<br />Right time<br />Speed-speed-speed<br />Right medium<br />Fact driven<br />Right message<br />Disciplined approach<br />Right frequency<br />Appropriate and Robust Toolset<br />Applicable to all needs<br />Acquisition, retention, conversion, growth/upsell<br />Success drivers and their prerequisites<br />
  43. 43. 250% increase in engagement<br />15% increase in new accounts<br />$1M increase in monthly revenue<br />12.7% increase in customer engagement<br />To drive results like…<br />Increased overall revenue per email message by 54%<br />187% increase in web site conversion <br />
  44. 44. A few thoughts on website and testing concepts<br />All concepts are not created equal<br /><ul><li>Discard bad ones as soon as you can
  45. 45. Testing bad concepts in different combinations and permutations is like rotating an ineffective player in different positions or formations in a sports team </li></ul>Don’t make customers learn your site<br /><ul><li>Follow industry standards as much as possible, unique may look good but may confuse customers
  46. 46. Unless some approach is critically important, visitor confusion </li></ul>Avoid the best of the worst<br /><ul><li>Unless concepts and ideas are grounded in strategy on you may be testing a few bad options and picking the least evil
  47. 47. Pick the optimal of the sensible</li></ul>Do not wait for perfection<br /><ul><li>Focus on pragmatic optimization, you don’t have time to wait for the perfect page, neither do your customers
  48. 48. Continual incremental improvement should be the goal- build a culture of testing and learning</li></li></ul><li>Four key strategies<br />Identify your true audience<br /><ul><li>Target through truly understanding and mapping your customer base to find trends and themes
  49. 49. Mine the information that already exists to build a more responsive position towards your customers</li></ul>Improve traction with customers<br /><ul><li>Create an intimate one-to-one dialog with each customer that maximizes conversion rates
  50. 50. Understand your current and prospective customers’ needs, and develops a long-term relationship to maximize lifetime customer value.</li></ul>Scalable and actionable testing<br /><ul><li>Ability to test the needed combinations of content quickly
  51. 51. Built against robust multidimensional segments based on a best-in-class methodology and a clear roadmap</li></ul>Direct impact for ROI Marketing<br /><ul><li>Targeting Cross-sell, up-sell, pipeline velocity, opportunity identification = improved ROI and Customer Lifetime Value
  52. 52. Testing of multiple channels against actual target audiences – measuring fiscal impact</li></li></ul><li>From<br />To<br />AnonymousUnknown and unmarketable based on broad audience<br />IdentifiableClearly defined customer road map that is directly connected to potential revenue and transactions<br />Media spendingArbitrary media investment based on best knowledge<br />Customer and experience investmentContent, connections and code that continually serves the customer through targeted experience<br />Mass offeringDesigned around defined target segments<br />Individual offerDesigned around unique needs of each and every target customer<br />Estimated marketing impactCorrelated or inferred advertising performance<br />Actual marketing impactTest / control scenarios, incremental lift and ROI analysis, constant optimization<br />Cursory customer insight collection<br />Continuous customer based monitoring and analysis<br />Make key shifts in approach…<br />
  53. 53. How does this happen?<br />Design a disciplined methodology<br />1. Establish <br />Strategic Objectives<br /><ul><li> Outline strategic objectives</li></ul>2. Establish Targets and Metrics<br /><ul><li> Define success metrics and create benchmarks
  54. 54. Define specific targets through deep-dive analytics</li></ul>6. Continuous Improvement<br /><ul><li> Put tracking mechanisms in place
  55. 55. Establish measurement methodology
  56. 56. Analyze results </li></ul>3. Track, Test, and Analyze<br /><ul><li>Identify and validate desired learning objectives for continuous improvement
  57. 57. Distribute meaningful results to the right teams</li></ul>4. Distribute Results<br /><ul><li> Use results and analytics for:
  58. 58. Development of ongoing actions
  59. 59. Strategy development & refinement</li></ul>5. Action Planning<br />
  60. 60. Analytics Tools<br />Tracks and report on visitor behavior<br />Tells us what's happening on our site<br />Trends visitor patterns allow for test/hypothesis<br />Tools are part of a program, not the full program<br />Program Optimization<br />Content delivery system with a statistical reporting interface<br />Tests which page performs best<br />vs<br />Makes recommendations that help improve the overall visitor experience<br />The goal is simple:<br />Organize information to actively listen and respond to your customers<br />
  61. 61. In Summary<br />All Analysis Begins with Business Objectives<br />Customer Data is Essentialto Optimization<br />Methodology is Thorough & Transparent<br />Insights are Based on Statistically Valid Metrics<br />Recommendations are Actionable & Impactful<br />
  62. 62. © 2010 Experian Information Solutions, Inc. All rights reserved.Experian Public.<br />24<br />Case study<br />
  63. 63. Starts with a question and a measurable outcome<br />A suitable DOE is constructed, test coded<br />Test experiences randomly exposed to traffic<br />Outcomes are measured <br />Results analyzed, decisions made <br />More questions… more tests…<br />Anatomy of a test<br />Traffic<br />Random split<br />A<br />Control<br />B<br />C<br />$$<br />$$$<br />$$$$<br />
  64. 64. Example<br />Challenge:<br /><ul><li>Numerous elements on page contributing to consumer product sales
  65. 65. Which elements have the greatest impact on results is unknown
  66. 66. Revenue increases needed quickly
  67. 67. Goal:
  68. 68. Increase the revenue per visitor by optimizing sales of consumer products</li></ul> U.S. home page<br />
  69. 69. Methodology:<br /><ul><li>Seven elements were tested with two variations each, yielding a potential 128 page versions
  70. 70. Using partial factorial test, eight-page versions were tested</li></ul>Test elements<br />6<br />1<br />2<br />3<br />7<br />4/5<br />Winning design @ 100%<br />
  71. 71. Anatomy of a test<br />eCommerce checkout process<br />Shopping cart<br />Register<br />Checkout<br />Review and submit<br />Order confirmation<br />– 57%<br />– 13%<br />– 12%<br />– 3%<br />
  72. 72. Anatomy of a test<br />eCommerce checkout process<br />Incorporate registration into shopping cart<br />
  73. 73. Anatomy of a text<br />eCommerce checkout process<br />Shopping cart<br />Checkout<br />Review and submit<br />Order confirmation<br />Register<br />– 32%<br />– 16%<br />– 3%<br />– 0.5%<br />
  74. 74. <ul><li>Significant reduction in drop off at key first step in the process
  75. 75. Overall visit to order rate for the site increased 3%
  76. 76. Significant differences in performance from new versus returning visitors
  77. 77. + 5% lift in visit to order rate for new visitors
  78. 78. – 18% lift in visit to order rate for returning visitors (this group was then eliminated from test group)
  79. 79. Type of product purchased did not impact performance</li></ul>Key findings<br />
  80. 80. Methodology:<br /><ul><li>Seven elements were tested with two variations each, yielding a potential 128 page versions
  81. 81. Using partial factorial test, eight-page versions were tested</li></ul>Test recap<br />Results: <br /><ul><li>+11% lift in overall eRPM
  82. 82. Main ad button (text), main ad text, image and transactional ad space were key drivers within this design</li></ul> daily eRPM<br />Winning design @ 100%<br />Test window@ 33%<br />
  83. 83. 138% lift in conversion rate (pre vs. post testing)<br />>100% increase in revenue / month (pre vs. post testing)<br />Why we continue to test<br />Results, results, results<br />
  84. 84. That showing multiple alternatives vs. a single version of content produces learning's and leads to data driven decisions<br />“Relevant content" is determined by our visitors and that not all visitors will prefer the same version of content<br />That the amount of testing we do and the speed at which we execute will have a significant impact on results<br />That creating a “dialog” with our visitors (based on their intent) is the only consistent way to increase online performance over time<br />Core tenets of testingWe have learned…<br />
  85. 85. Q&A<br />
  86. 86. Speaker Contact Info. <br /><ul><li>
  87. 87. 904. 217.4801 </li></ul>36<br />Thank You/Credit Slide<br />
  88. 88. 37<br />Thank You<br />Visit<br /><br />for more information<br /> Follow us @OMSummit<br />