Scorecards - So You Can Focus On Analysis
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Scorecards - So You Can Focus On Analysis

  • 1,541 views
Uploaded on

Kiki Hager, VML, ...

Kiki Hager, VML,

Traditional software vendors have failed to develop technology in a way that feeds into decision-making processes. Ogilvy fills this void and with a leading online and offline dashboard solution with innovative and powerful reporting for financial services, logistic and CPG clients.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,541
On Slideshare
1,525
From Embeds
16
Number of Embeds
1

Actions

Shares
Downloads
58
Comments
0
Likes
0

Embeds 16

http://engage.webtrends.com 16

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Omari JinakiKiki HagerScorecards - So You Can Focus On Analysis
  • 2. Introductions
    Ogilvy
    • In 1948, David Ogilvy founded the agency that would become Ogilvy & Mather. Starting with no clients and a staff of two, he built his company into one of the eight largest advertising networks in the world. Today it has more than 450 plus offices in 169 cities.
    • 3. Ogilvy has over 20-years of experience in developing, refining, and deploying analytics. Ogilvy’s Global Data and Analytics practice is one of the most sophisticated data consultancies in the industry with over 250 professionals at hubs in New York, Singapore, Beijing, Sao Paulo, London, Paris and Frankfurt.
    • 4. Our mission: turn data into increased accountability, insight, creativity and action through 4 key capabilities: Measurement, Optimization, Insights & Data Planning.
    • 5. Rated #1 by Forrester with capabilities in capabilities include a breadth of data mining, web analytics, statistical modeling, media analytics, econometric optimization, and business intelligence.
  • Introductions
    VML
    • Founded in 1992, VML is a digital marketing agency based in Kansas City, Missouri. We have grown to more than 800 people around the globe. Key client partners include Gatorade, Sam’s Club, Ford of Canada, Hershey’s and Microsoft. VML was named as a leader in the 2009 Forrester Wave report, ranked number one in Current Offering, including the Top Score for Strategy.
    • 6. The VML Analytics and Optimization team provides unmatched experience implementing, managing, analyzing and optimizing online programs and sites ranging from content driven portals to best in class ecommerce storefronts. Our team, headed by Eric Beane, is comprised of specialists across multiple analytic packages including certifications in Webtrends, Omniture Site Catalyst, Yahoo Web Analytics and Google Analytics. We are also experts in A/B, Multivariate testing leading our clients to double digit revenue growth.
    Kiki Hager
    Web Analyst, VML
    khager@vml.com
    816 218 3097
  • 7. Scorecards - So You Can Focus On Analysis
    3 pillars of a scorecard (this presentation will cover scorecard design and execution)
    Set Objectives
    • Define your business objectives: Why do you need a scorecard? What business questions will it answer?
    Design
    • Use 9 best practices to design a smart scorecard
    Execute
    • Automate your reporting; 2 case studies around design and automation (Emerson and an Agricultural Company)
  • Scorecards - Designing a smart scorecard
    • 9 best practicesto designing a scorecard. Ensure you can focus on analysis and deliver on your business objectives.
    Caveat!
    • not all scorecards require all the steps
    • 8. depends on your/clients requirements
    1
    Appraise your data
    2
    Assemble data
    3
    Select the right metrics
    4
    Set expectations
    5
    Claim success
    6
    Feature results
    7
    Inject Insights
    8
    Generate a score
    9
    Automate
  • 9. Rethink scorecard design
  • 10. Appraise your data
    • Knowing what data is valuable is much more important than getting the data.
    • 11. More data, more metrics, are promised from more sources. A rich media unit generates over 250 data points, web analytics data comes in large volumes and requires complex systems to collect and manage.
    Know your data sources and how your data is collected
    • Understand the sources and be intimate with your website implementation
    • 12. Not all data sources are created equal, evaluate each data source by ease of access and value
  • Assemble data
    • Data structure varies by source. Integrating data, especially data across sources, and channels is not easy to do.
    • 13. Tracking 360 marketing activity will only be possible through data integration. Organizations that start facing up to these challenges will be best placed for success.
    • 14. Often the time dimension is the lowest common factor.
    Combine and normalize key data across the relevant channels
    • Align data by the lowest common denominator
    • 15. Combining data will enable the calculation of cross channel metrics
    Aggregate
  • 16. Select the right metrics
    • A scorecard is only as meaningful as the metrics selected. Distinguish between operational metrics and performance metrics. Performance should be directly aligned with your business objectives.
    • 17. Go beyond tracking just the basic performance metrics, develop a two tiered tracking system of Leading and Conversion metrics (a.k.a. key performance metrics - KPMs).
    Align metrics to your objectives, separate out operational metrics
    • Report on metrics that matter, leverage leading metrics as key indicators
    • 18. Operational metrics will track the execution of your campaign, program, etc.
    Objective
    Performance Metric
    Operational Metric
    Unique Visits
    Awareness
    Bounce rate
    L
    Downloads
    Content engagement
    Views
    L
    Registrations
    Leads
    Fields completed
    K
    Revenue
    Transaction
    Thank you page
    K
  • 19. Set expectations
    • State what is to be achieved when results meet/exceed expectations.
    • 20. Evaluating results is a relative exercise unless compared to a benchmark. A benchmark (4 approaches) should be set to be achievable, have a set timeline, and most importantly be based on sound assumptions and math.
    Design a forecast for your key performance metrics, prior!
    • Performance metrics should drive optimization and strategy shifts
    • 21. Operational metrics will track the execution of your campaign, program, etc.
    Four approaches to set expectations:
    Forecast: Forecast performance for each KPM
    Syndicated Benchmarks: Leverage syndicated and/or competitive performance outcomes
    Pre/Post Comparison: Compare in-market performance, before and after a launch
    Test/Control : Set aside a control environment to compare results of in-market performance
  • 22. Claim success
    • Improvement in your KPMs is great; but is it real and replicable?
    • 23. Provide reliability by ensuring statistical confidence when comparing results. Basing your decisions on untested results may lead you down the wrong/unreliable path.
    Ensure statistical rigor when comparing results, 95%+ confidence
    • Sample size is dependent on the variance of results (small improvements are hard to prove)
    • 24. Choosing the right statistic is a science, a different approach can lead to a different conclusion
    Upper Bound
    Confidence Intervals
    No difference in change
    Lower Bound
  • 25. Feature results
    • Know your audience! Understand their needs!
    • 26. This will provide a good sense of how the reader will visualize your scorecards, your charts, and graphical elements.
    • 27. It is crucial to be clear in explaining your results. Be clear in what the numbers represent and present the data in a manner that allows the reader to quickly grasp what the results mean.
    Create output which is self-explanatory, styled, and easy to understand
    • Your output should facilitate quick assessment
    • 28. Visualize data to perceive your message at a glance, although numbers will drive decisions
    Raw data to results
  • 29. Inject insights
    • What do the results mean?
    • 30. Squeeze insights out of your results. Manipulate/triangulate your data, compare trends, test and challenge hypotheses – in the end, translate results into meaningful transparent insights to be used for driving decisions and agile strategies.
    Translate results into evidence-based insights
    • Deliver insights that are gleaned from data
    • 31. Always have a “deep-dive” or specific area of more detailed analysis within your reports
    Results to insights
  • 32. Generate a score
    • Get an overall directional sense of performance.
    • 33. A visitor score is derived by assigning a relative value to individual visitors based on their actions/behaviors over time. You can determine intent and engagement – even before visitors convert.
    • 34. If the score is calibrated in just right way, you can compare across sites, campaigns, and programs.
    Leverage simplicity of a score without sacrificing detail
    • Intuitive weighting: assumptions based on valued activities, using business acumen
    • 35. Empirical weighting: values derived through statistical modeling
  • Automate
    • Don’t be a trained monkey, don’t be late with results!
    • 36. Results have a shelf-life. To be an agile marketer you need to act and adjust quickly – dated results will put you behind the curve.
    • 37. Outside of timely results, automation will reduce cost, introduce reliability, and provide a documented process of delivering the necessary information in the right format at the right time.
    Automate your reporting to be a repeatable, reliable process
    • Enable the end-user to leverage fresh results/insights to make better decisions
    • 38. Leverage automation to save time and money; free-up your resources for analysis
    There are multiple ways that you get data into Excel using the REST URL APIs.  It is fairly straightforward process to use it to get web data and populate rows of data where you want them.  Two very simple ways to do this, of course, are: 1) use the REST URL Generator application available at https://generator.webtrends.com; and 2) to use REST URLs that are generated with the new Webtrends Analytics 9 Insight interface (http://insight.webtrends.com).
  • 39. Execution – Emerson Case Study
    • Situation: Emerson Network Power is a global leader in industry leading technologies and solutions for data centers, IT and telecommunications networks. Headquartered in Columbus, OH, Emerson employs over 43,000 people worldwide. In August 2010, Emerson launched a site to incorporate CMS capabilities and more in-depth analytics. While they had been running Webtrends On Demand, they had not actively been utilizing the data and reports available to them.
    • 40. Objective: Determine a way to dynamically gather data across multiple sites and multiple profiles from Webtrends. This report will be used by multiple international owners across nine brands, each with a unique Webtrends set up and reporting structure.
    • 41. Strategy: Talk to the end users of the scorecards, find out what is currently being reported on, what they would change, and how they use data to make decisions.
    • 42. Tactics: Create a scorecard that contains multiple tabs for different needs / viewing levels (monthly vs. custom date ranges). Utilizing Webtrends REST URLs, and the VBA Sample Application built by Michael Love, we will be able to produce a dynamic Excel Workbook that fits the needs across all brands and marketing owners. For ease of use, we will add drop downs for profile selections and date ranges, and dynamically populate the data returned across multiple tabs and reports.
  • Execution – Emerson Case Study
    The control tab within the report fetches the data based on the user defined credentials. It allows the user to select their profile and their desired date range. This is where all of the REST URL’s are also running behind the scene (they are hidden from the users view).
    The data tab maintains historical data for analyzing trends over time, as well as the data repository for other sheets and reports.
    1
    2
    1
    2
  • 43. Execution – Emerson Case Study
    The Monthly Scorecard tab contains key metrics that provide a high level overview of the health of the sites. It offers drop downs for easy month over month comparisons and is updated based on current selections, including all charts, tables, and graphs.
    Key Metrics / Conversion Points
  • 44. Execution – Emerson Case Study
    The Custom Date Range Scorecard tab contains more detailed metrics that drill deeper into the data based on the Custom Date range selected.
    Suggested Metrics:Search Engines, Search Phrases, Regions, Referring Domains, Top Pages
  • 45. Execution – Emerson Case Study
    Upon delivering the scorecards to each stakeholder, feedback has been outstanding. It has been flexible and easy to use, but more importantly Emerson brand owners have been able to accomplish some key tweaks to their site / media strategy based on the data they are able to retrieve.
    These impacts included the following:
    • Quickly quantified campaign success
    • 46. Made content changes based on the top pages report
    • 47. Modified keyword purchases
    • 48. Identified an error on the site by seeing drastic drops in traffic and were able to make the correction and get the site back up and running
    A copy of this dashboard can be found at the URL below:
    http://winstage.vml.com/EngageConference/Dashboard/Engage_Conference_021611.xlsm
  • 49. Execution – Agricultural Company Case Study
    • Situation: With offices around the world; this company produces, markets and sells hybrid seed corn in nearly 70 countries worldwide. They had been running old versions of Webtrends Software for several years, without SDC. In November of 2010, they launched a new site and with that updated their Webtrends version and incorporated SDC, so now they had accurate data but didn’t know how to consume it.
    • 50. Objective: Create a dashboard to be used by the Marketing Manager that is visual and appealing to someone who does not have a data background. The report needs to be updatable on the fly, but can’t utilize REST URLs because the client doesn’t have the latest software version.
    • 51. Strategy: Start with the KPIs, then develop a “fake” dashboard that doesn’t have any API functionality. This gives the stakeholder something to respond to without doing all the excel / VBA heavy lifting. Once the format is agreed to, we will build the back end using ODBC.
    • 52. Tactics: Create a multi tab dashboard that provides an Executive Summary for the Marketing Manager with many visuals including graphs, charts, and multiple note fields where actionable recommendations will be placed. We will then train the client on how to install the ODBC driver and how to update the dashboard themselves.
  • Execution – Agricultural Company Case Study
    The input tab within the report fetches the data based on the selected profile and date range. It utilizes an ODBC driver and VBA code to gather and update data.
    Similar to the data tab on the Emerson Case Study, the data tab in the ODBC report maintains historical data for analyzing trends over time, as well as the data repository for other sheets and reports.
    1
    2
    1
    2
  • 53. Execution – Agricultural Company Case Study
    The Executive Scorecard tab contains key metrics that provide a high level overview of traffic to the site. It offers many visuals for quick analysis and interpretation.
    Key Metrics
  • 54. Execution – Pioneer Case Study
    The Content / Sub Content Group tab takes advantage of one of the key features in Webtrends and provides a report that is high level and easy to read.
    Suggested Metrics:Visits / Page Views / Page Views/Visit / Average Time Viewed
  • 55. Execution – Agricultural Company Case Study
    The Monthly Topline tab is used to compare year over year trending to determine how seasonality affects traffic.
    Key Metrics
  • 56. Execution – Agricultural Company Case Study
    The executive team was able to put the dashboard to immediate use by realizing their visitors are 85% returning, so they have ramped up their paid media efforts to expand their consumer base.
    Understanding they want to broaden their reach, they were, however, thrilled with the retention level of the new site, as it was built to be a tool that farmers could make their own. By identifying their location, the site will display their local weather, sales team, bills / payment information, local seed prices in their area, and many other custom tools meant to keep up with the farmers of tomorrow. The dashboard confirms their strategy is working!
  • 57. Checking off on all 9 Best Practices
  • 58. Execution – Sample Scorecards