Your SlideShare is downloading. ×
Online Test, Target and Measurement - Nancy Shaver, Experian Marketing Services
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Online Test, Target and Measurement - Nancy Shaver, Experian Marketing Services


Published on

Online Test, Target and Measurement …

Online Test, Target and Measurement

The pressure to deliver results and show ROI for all marketing spend is further complicated by market pace changes and economic climate. Learn how to find the optimal mix of ideas, and do it faster to increase ROI on marketing spend and provide relevant content and interactions to target audiences through the different channels including web, email, and mail.

* Nancy Shaver, Consulting Principal, Experian Marketing Services

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Online Marketing Summit
    Washington DC, July 2010
    a world of insight
    Online Test, Target and Measurement
    Nancy Shaver
    Consulting Principal
    Marketing AnalyticsExperian Marketing Services
  • 2. Bio
    Nancy Shaver is a Consulting Principal for Experian Marketing Solutions’ Marketing Analytics team. In this role, she leverages both her strategic vision and extensive hands-on direct, digital and database marketing experience to help Experian’s clients develop and implement achievable marketing solutions that deliver measurable business value. Through a combination of her work across a range of clients including retail, telecommunications, consumer packaged goods, and financial services, as well as her deep operational and strategy experience, Nancy is able to help clients translate their goals into strategies and practical and effective initiatives which link a positive customer experience with bottom line results.
    Nancy has over 25 years of experience leading marketing organizations as well as delivering data, analytical, and technical services. Prior to joining Experian, Nancy was a principal and founder of 6C Consulting, specializing in optimizing multichannel customer marketing effectiveness. She has also held senior management positions at MCI and Claritas. Other experience includes participation in a campaign management software start-up, as well as extensive account and operations management at a marketing service provider.
    Nancy has been a speaker at industry conferences and has been a board member of the DMAW Educational Foundation and TransAmerica Marketing. Client relationships have included several of Experian’s large strategic accounts.
    Nancy holds a Bachelor of Arts from Wellesley College and attended the MBA program at Mary Washington University.
  • 3.
    • Market Situation
    • 4. Challenges… and opportunities
    • 5. Its all in the approach
    • 6. Drinking the Koolaid
    Session agenda
  • 7. Experian overview
    Marketing Services
    Credit Services
  • 8. Part of $3.8 billion Experian
    Experian Marketing Services moves marketing forward with predictive insights that improve how marketers target and engage their customers across both traditional and digital media
    Marketing Services
    Credit Services
    Decision Analytics
    Experian Marketing Services
  • 9. Unparalleled data assets that give a 360º customer view
    Strategic, data-driven expertise to optimize and grow programs
    Technology to target, segment and engage across channels
    • Industry's largest supply of consumer data
    • 10. Data includes:Demographics, attitudes, lifestyles, media habits, purchases, auto and credit scores, internet usage, life events, etc.
    • 11. Rich history as secure, trusted data provider
    • 12. Over 30 years of proven analytical expertise
    • 13. Analysis to understand past marketing activities
    • 14. Predictive models toidentify future success
    • 15. Establish business strategies to optimize programs and ensure continued growth
    • 16. Best-in-breed email marketing for enterprise
    • 17. Cutting edge real-time consumer profiling on web, mobile, TV
    • 18. True multi-channel data integration
    • 19. Industry leading address verification & datahygiene tools
    Core expertise
    Experian Marketing Services
  • 20. Q1 – Why did the French team crash out?
    Q2 – Why did the Italian team crash out?
    Q3 – Why did the England team crash out?
    Q3 – Why did Spain win the World Cup?
    Quick survey
    World Cup
  • 21. Demand for measurable returns have accelerated
    Channel proliferation continues
    Elevated consumer expectations
    Program performance measures are complex and challenging
    Historical marketing “silos” make integration difficult
    Where we are
    New Marketing Ecosystem
  • 22. Trends in the marketplace
    CMO Council’s latest State of Marketing report
  • 23. Trends in the marketplace
    CMO Council’s latest State of Marketing report
  • 24. Digital landscape
  • 25. External
    Marketers challenges
    External factors
    • Media breadth and influence continually shifting
    • 26. Pervasiveness of technology
    Shifting customer dynamics
    • Demographic and lifestyle changes
    • 27. Increase in the level of choice within customer viewpoint
    Marketplace changes
    • Current climate altering consumer behavior
    • 28. Rapidly changing marketing conditions
    • 29. Move towards global markets increasing
    Data growth
    • More information sources – external and internal fragmentation
    • 30. Discipline and time to absorb in granular nature difficult
    Internal obstacles
    • Staffs are over-worked, limited budget for add’l FTE
    • 31. Building a COE not realistically feasible within timeframes
    ROI based decision making
    • Shift to accountable marketing and scrutiny
    • 32. Expectations of doing move with limited budgets
  • Market place situation
    Current needs – a mixture of old and new
    How do I segment and target my prospects in the new media environment?
    Can’t we use purchase history to tell me what I should be offering each customer?
    We have to cut circulation budgets. How do we do that while preserving revenue?
    How do we correctly attribute sales to direct mail, email, web and media?
    We’ve churned through a lot of consumers. How do I get the most value out of my formers?
    How do I optimize contacts across channels - direct mail, email and web?
    We’ve optimized response of each campaign. How do I optimize the value of the customer to us?
    I’m losing as many customers as I acquire every year. What can I do?
  • 33. Market place situation requires a shift
    Defensive posture
    • A focus on customer programs
    • 34. Reduced scope, scale and spend
    • 35. Optimization, a common core theme
    Change continues
    • Competitive position and industry players
    • 36. Consumer attitudes, behaviors and media consumption
    • 37. Customer aging
    • 38. Digital technology and media advances
    Analytic refresh and upgrade cycle
    • Segmentation systems and profiles reflecting who customers were, not who they are
    • 39. Out-of-date acquisition strategies and prospect models
    • 40. Customer targeting systems that do not reflect changed customer expectations
    Shifting to pro-active position has become more complex than just having a plan – it is imperative understand and assimilate all information and channels available
  • 41. What you need to know…
    …to plan to address their needs:
    What each customer buys?
    Who exactly are my customers?
    Where are they outside of your brand experience?
    Customer Lifetime Value
    Where each customer is all the time?
    Their cross-channel usage
    Are they your top brand advocates?
    Two-way communication with the customer
    Top of mind competitors
    Click-stream information
    What is the right marketing investment for them?
    Deep segmentation and understanding
    Customer Satisfaction and Loyalty
    Why do they buy from you, and how do they buy from you?
  • 42. Applies to all channels
    Right audience
    Customer intimacy
    Right time
    Right medium
    Fact driven
    Right message
    Disciplined approach
    Right frequency
    Appropriate and Robust Toolset
    Applicable to all needs
    Acquisition, retention, conversion, growth/upsell
    Success drivers and their prerequisites
  • 43. 250% increase in engagement
    15% increase in new accounts
    $1M increase in monthly revenue
    12.7% increase in customer engagement
    To drive results like…
    Increased overall revenue per email message by 54%
    187% increase in web site conversion
  • 44. A few thoughts on website and testing concepts
    All concepts are not created equal
    • Discard bad ones as soon as you can
    • 45. Testing bad concepts in different combinations and permutations is like rotating an ineffective player in different positions or formations in a sports team
    Don’t make customers learn your site
    • Follow industry standards as much as possible, unique may look good but may confuse customers
    • 46. Unless some approach is critically important, visitor confusion
    Avoid the best of the worst
    • Unless concepts and ideas are grounded in strategy on you may be testing a few bad options and picking the least evil
    • 47. Pick the optimal of the sensible
    Do not wait for perfection
    • Focus on pragmatic optimization, you don’t have time to wait for the perfect page, neither do your customers
    • 48. Continual incremental improvement should be the goal- build a culture of testing and learning
  • Four key strategies
    Identify your true audience
    • Target through truly understanding and mapping your customer base to find trends and themes
    • 49. Mine the information that already exists to build a more responsive position towards your customers
    Improve traction with customers
    • Create an intimate one-to-one dialog with each customer that maximizes conversion rates
    • 50. Understand your current and prospective customers’ needs, and develops a long-term relationship to maximize lifetime customer value.
    Scalable and actionable testing
    • Ability to test the needed combinations of content quickly
    • 51. Built against robust multidimensional segments based on a best-in-class methodology and a clear roadmap
    Direct impact for ROI Marketing
    • Targeting Cross-sell, up-sell, pipeline velocity, opportunity identification = improved ROI and Customer Lifetime Value
    • 52. Testing of multiple channels against actual target audiences – measuring fiscal impact
  • From
    AnonymousUnknown and unmarketable based on broad audience
    IdentifiableClearly defined customer road map that is directly connected to potential revenue and transactions
    Media spendingArbitrary media investment based on best knowledge
    Customer and experience investmentContent, connections and code that continually serves the customer through targeted experience
    Mass offeringDesigned around defined target segments
    Individual offerDesigned around unique needs of each and every target customer
    Estimated marketing impactCorrelated or inferred advertising performance
    Actual marketing impactTest / control scenarios, incremental lift and ROI analysis, constant optimization
    Cursory customer insight collection
    Continuous customer based monitoring and analysis
    Make key shifts in approach…
  • 53. How does this happen?
    Design a disciplined methodology
    1. Establish
    Strategic Objectives
    • Outline strategic objectives
    2. Establish Targets and Metrics
    • Define success metrics and create benchmarks
    • 54. Define specific targets through deep-dive analytics
    6. Continuous Improvement
    • Put tracking mechanisms in place
    • 55. Establish measurement methodology
    • 56. Analyze results
    3. Track, Test, and Analyze
    • Identify and validate desired learning objectives for continuous improvement
    • 57. Distribute meaningful results to the right teams
    4. Distribute Results
    • Use results and analytics for:
    • 58. Development of ongoing actions
    • 59. Strategy development & refinement
    5. Action Planning
  • 60. Analytics Tools
    Tracks and report on visitor behavior
    Tells us what's happening on our site
    Trends visitor patterns allow for test/hypothesis
    Tools are part of a program, not the full program
    Program Optimization
    Content delivery system with a statistical reporting interface
    Tests which page performs best
    Makes recommendations that help improve the overall visitor experience
    The goal is simple:
    Organize information to actively listen and respond to your customers
  • 61. In Summary
    All Analysis Begins with Business Objectives
    Customer Data is Essentialto Optimization
    Methodology is Thorough & Transparent
    Insights are Based on Statistically Valid Metrics
    Recommendations are Actionable & Impactful
  • 62. © 2010 Experian Information Solutions, Inc. All rights reserved.Experian Public.
    Case study
  • 63. Starts with a question and a measurable outcome
    A suitable DOE is constructed, test coded
    Test experiences randomly exposed to traffic
    Outcomes are measured
    Results analyzed, decisions made
    More questions… more tests…
    Anatomy of a test
    Random split
  • 64. Example
    • Numerous elements on page contributing to consumer product sales
    • 65. Which elements have the greatest impact on results is unknown
    • 66. Revenue increases needed quickly
    • 67. Goal:
    • 68. Increase the revenue per visitor by optimizing sales of consumer products U.S. home page
  • 69. Methodology:
    • Seven elements were tested with two variations each, yielding a potential 128 page versions
    • 70. Using partial factorial test, eight-page versions were tested
    Test elements
    Winning design @ 100%
  • 71. Anatomy of a test
    eCommerce checkout process
    Shopping cart
    Review and submit
    Order confirmation
    – 57%
    – 13%
    – 12%
    – 3%
  • 72. Anatomy of a test
    eCommerce checkout process
    Incorporate registration into shopping cart
  • 73. Anatomy of a text
    eCommerce checkout process
    Shopping cart
    Review and submit
    Order confirmation
    – 32%
    – 16%
    – 3%
    – 0.5%
  • 74.
    • Significant reduction in drop off at key first step in the process
    • 75. Overall visit to order rate for the site increased 3%
    • 76. Significant differences in performance from new versus returning visitors
    • 77. + 5% lift in visit to order rate for new visitors
    • 78. – 18% lift in visit to order rate for returning visitors (this group was then eliminated from test group)
    • 79. Type of product purchased did not impact performance
    Key findings
  • 80. Methodology:
    • Seven elements were tested with two variations each, yielding a potential 128 page versions
    • 81. Using partial factorial test, eight-page versions were tested
    Test recap
    • +11% lift in overall eRPM
    • 82. Main ad button (text), main ad text, image and transactional ad space were key drivers within this design daily eRPM
    Winning design @ 100%
    Test window@ 33%
  • 83. 138% lift in conversion rate (pre vs. post testing)
    >100% increase in revenue / month (pre vs. post testing)
    Why we continue to test
    Results, results, results
  • 84. That showing multiple alternatives vs. a single version of content produces learning's and leads to data driven decisions
    “Relevant content" is determined by our visitors and that not all visitors will prefer the same version of content
    That the amount of testing we do and the speed at which we execute will have a significant impact on results
    That creating a “dialog” with our visitors (based on their intent) is the only consistent way to increase online performance over time
    Core tenets of testingWe have learned…
  • 85. Q&A
  • 86. Speaker Contact Info.
    • 87. 904. 217.4801
    Thank You/Credit Slide
  • 88. 37
    Thank You
    for more information
    Follow us @OMSummit