Omari JinakiKiki HagerScorecards - So You Can Focus On Analysis<br />
Introductions<br />Ogilvy<br /><ul><li>In 1948, David Ogilvy founded the agency that would become Ogilvy & Mather. Startin...
Ogilvy has over 20-years of experience in developing, refining, and deploying analytics. Ogilvy’s Global Data and Analytic...
Our mission: turn data into increased accountability, insight, creativity and action through 4 key capabilities: Measureme...
Rated #1 by Forrester with capabilities in capabilities include a breadth of data mining, web analytics, statistical model...
The VML Analytics and Optimization team provides unmatched experience implementing, managing, analyzing and optimizing onl...
Scorecards - So You Can Focus On Analysis<br />3 pillars of a scorecard (this presentation will cover scorecard design and...
 depends on your/clients requirements</li></ul>1<br />Appraise your data<br />2<br />Assemble data<br />3<br />Select the ...
Rethink scorecard design<br />
Appraise your data<br /><ul><li> Knowing what data is valuable is much more important than getting the data.
 More data, more metrics, are promised from more sources.  A rich media unit generates over 250 data points, web analytics...
 Not all data sources are created equal, evaluate each data source by ease of access and value</li></li></ul><li>Assemble ...
 Tracking 360 marketing activity will only be possible through data integration. Organizations that start facing up to the...
 Often the time dimension is the lowest common factor.</li></ul>Combine and normalize key data across the relevant channel...
 Combining data will enable the calculation of cross channel metrics</li></ul>Aggregate<br />
Select the right metrics<br /><ul><li> A scorecard is only as meaningful as the metrics selected. Distinguish between oper...
 Go beyond tracking just the basic performance metrics, develop a two tiered tracking system of Leading and Conversion met...
 Operational metrics will track the execution of your campaign, program, etc.</li></ul>Objective<br />Performance Metric<b...
Set expectations<br /><ul><li> State what is to be achieved when results meet/exceed expectations.
 Evaluating results is a relative exercise unless compared to a benchmark.  A benchmark (4 approaches) should be set to be...
 Operational metrics will track the execution of your campaign, program, etc.</li></ul>Four approaches to set expectations...
Claim success<br /><ul><li> Improvement in your KPMs is great; but is it real and replicable?
 Provide reliability by ensuring statistical confidence when comparing results.  Basing your decisions on untested results...
 Choosing the right statistic is a science, a different approach can lead to a different conclusion</li></ul>Upper Bound<b...
Feature results<br /><ul><li> Know your audience!  Understand their needs!
 This will provide a good sense of how the reader will visualize your scorecards, your charts, and graphical elements.
 It is crucial to be clear in explaining your results. Be clear in what the numbers represent and present the data in a ma...
 Visualize data to perceive your message at a glance, although numbers will drive decisions</li></ul>Raw data to results<b...
Inject insights<br /><ul><li> What do the results mean?
Upcoming SlideShare
Loading in...5
×

Scorecards - So You Can Focus On Analysis

1,309

Published on

Kiki Hager, VML,

Traditional software vendors have failed to develop technology in a way that feeds into decision-making processes. Ogilvy fills this void and with a leading online and offline dashboard solution with innovative and powerful reporting for financial services, logistic and CPG clients.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,309
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
63
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Scorecards - So You Can Focus On Analysis

  1. 1. Omari JinakiKiki HagerScorecards - So You Can Focus On Analysis<br />
  2. 2. Introductions<br />Ogilvy<br /><ul><li>In 1948, David Ogilvy founded the agency that would become Ogilvy & Mather. Starting with no clients and a staff of two, he built his company into one of the eight largest advertising networks in the world. Today it has more than 450 plus offices in 169 cities.
  3. 3. Ogilvy has over 20-years of experience in developing, refining, and deploying analytics. Ogilvy’s Global Data and Analytics practice is one of the most sophisticated data consultancies in the industry with over 250 professionals at hubs in New York, Singapore, Beijing, Sao Paulo, London, Paris and Frankfurt.
  4. 4. Our mission: turn data into increased accountability, insight, creativity and action through 4 key capabilities: Measurement, Optimization, Insights & Data Planning.
  5. 5. Rated #1 by Forrester with capabilities in capabilities include a breadth of data mining, web analytics, statistical modeling, media analytics, econometric optimization, and business intelligence.</li></li></ul><li>Introductions<br />VML<br /><ul><li>Founded in 1992, VML is a digital marketing agency based in Kansas City, Missouri. We have grown to more than 800 people around the globe. Key client partners include Gatorade, Sam’s Club, Ford of Canada, Hershey’s and Microsoft. VML was named as a leader in the 2009 Forrester Wave report, ranked number one in Current Offering, including the Top Score for Strategy.
  6. 6. The VML Analytics and Optimization team provides unmatched experience implementing, managing, analyzing and optimizing online programs and sites ranging from content driven portals to best in class ecommerce storefronts. Our team, headed by Eric Beane, is comprised of specialists across multiple analytic packages including certifications in Webtrends, Omniture Site Catalyst, Yahoo Web Analytics and Google Analytics. We are also experts in A/B, Multivariate testing leading our clients to double digit revenue growth. </li></ul>Kiki Hager<br />Web Analyst, VML<br />khager@vml.com<br />816 218 3097<br />
  7. 7. Scorecards - So You Can Focus On Analysis<br />3 pillars of a scorecard (this presentation will cover scorecard design and execution) <br />Set Objectives<br /><ul><li>Define your business objectives: Why do you need a scorecard? What business questions will it answer?</li></ul>Design <br /><ul><li>Use 9 best practices to design a smart scorecard</li></ul>Execute <br /><ul><li>Automate your reporting; 2 case studies around design and automation (Emerson and an Agricultural Company)</li></li></ul><li>Scorecards - Designing a smart scorecard<br /><ul><li>9 best practicesto designing a scorecard. Ensure you can focus on analysis and deliver on your business objectives.</li></ul>Caveat!<br /><ul><li> not all scorecards require all the steps
  8. 8. depends on your/clients requirements</li></ul>1<br />Appraise your data<br />2<br />Assemble data<br />3<br />Select the right metrics<br />4<br />Set expectations<br />5<br />Claim success<br />6<br />Feature results<br />7<br />Inject Insights<br />8<br />Generate a score<br />9<br />Automate<br />
  9. 9. Rethink scorecard design<br />
  10. 10. Appraise your data<br /><ul><li> Knowing what data is valuable is much more important than getting the data.
  11. 11. More data, more metrics, are promised from more sources. A rich media unit generates over 250 data points, web analytics data comes in large volumes and requires complex systems to collect and manage. </li></ul>Know your data sources and how your data is collected<br /><ul><li> Understand the sources and be intimate with your website implementation
  12. 12. Not all data sources are created equal, evaluate each data source by ease of access and value</li></li></ul><li>Assemble data<br /><ul><li> Data structure varies by source. Integrating data, especially data across sources, and channels is not easy to do.
  13. 13. Tracking 360 marketing activity will only be possible through data integration. Organizations that start facing up to these challenges will be best placed for success.
  14. 14. Often the time dimension is the lowest common factor.</li></ul>Combine and normalize key data across the relevant channels<br /><ul><li> Align data by the lowest common denominator
  15. 15. Combining data will enable the calculation of cross channel metrics</li></ul>Aggregate<br />
  16. 16. Select the right metrics<br /><ul><li> A scorecard is only as meaningful as the metrics selected. Distinguish between operational metrics and performance metrics. Performance should be directly aligned with your business objectives.
  17. 17. Go beyond tracking just the basic performance metrics, develop a two tiered tracking system of Leading and Conversion metrics (a.k.a. key performance metrics - KPMs). </li></ul>Align metrics to your objectives, separate out operational metrics<br /><ul><li> Report on metrics that matter, leverage leading metrics as key indicators
  18. 18. Operational metrics will track the execution of your campaign, program, etc.</li></ul>Objective<br />Performance Metric<br />Operational Metric<br />Unique Visits<br />Awareness<br />Bounce rate<br />L<br />Downloads<br />Content engagement<br />Views<br />L<br />Registrations<br />Leads<br />Fields completed<br />K<br />Revenue<br />Transaction<br />Thank you page<br />K<br />
  19. 19. Set expectations<br /><ul><li> State what is to be achieved when results meet/exceed expectations.
  20. 20. Evaluating results is a relative exercise unless compared to a benchmark. A benchmark (4 approaches) should be set to be achievable, have a set timeline, and most importantly be based on sound assumptions and math.</li></ul>Design a forecast for your key performance metrics, prior!<br /><ul><li> Performance metrics should drive optimization and strategy shifts
  21. 21. Operational metrics will track the execution of your campaign, program, etc.</li></ul>Four approaches to set expectations:<br />Forecast: Forecast performance for each KPM<br />Syndicated Benchmarks: Leverage syndicated and/or competitive performance outcomes<br />Pre/Post Comparison: Compare in-market performance, before and after a launch<br />Test/Control : Set aside a control environment to compare results of in-market performance<br />
  22. 22. Claim success<br /><ul><li> Improvement in your KPMs is great; but is it real and replicable?
  23. 23. Provide reliability by ensuring statistical confidence when comparing results. Basing your decisions on untested results may lead you down the wrong/unreliable path.</li></ul>Ensure statistical rigor when comparing results, 95%+ confidence<br /><ul><li> Sample size is dependent on the variance of results (small improvements are hard to prove)
  24. 24. Choosing the right statistic is a science, a different approach can lead to a different conclusion</li></ul>Upper Bound<br />Confidence Intervals<br />No difference in change<br />Lower Bound<br />
  25. 25. Feature results<br /><ul><li> Know your audience! Understand their needs!
  26. 26. This will provide a good sense of how the reader will visualize your scorecards, your charts, and graphical elements.
  27. 27. It is crucial to be clear in explaining your results. Be clear in what the numbers represent and present the data in a manner that allows the reader to quickly grasp what the results mean. </li></ul>Create output which is self-explanatory, styled, and easy to understand<br /><ul><li> Your output should facilitate quick assessment
  28. 28. Visualize data to perceive your message at a glance, although numbers will drive decisions</li></ul>Raw data to results<br />
  29. 29. Inject insights<br /><ul><li> What do the results mean?
  30. 30. Squeeze insights out of your results. Manipulate/triangulate your data, compare trends, test and challenge hypotheses – in the end, translate results into meaningful transparent insights to be used for driving decisions and agile strategies. </li></ul>Translate results into evidence-based insights<br /><ul><li> Deliver insights that are gleaned from data
  31. 31. Always have a “deep-dive” or specific area of more detailed analysis within your reports</li></ul>Results to insights<br />
  32. 32. Generate a score<br /><ul><li> Get an overall directional sense of performance.
  33. 33. A visitor score is derived by assigning a relative value to individual visitors based on their actions/behaviors over time. You can determine intent and engagement – even before visitors convert.
  34. 34. If the score is calibrated in just right way, you can compare across sites, campaigns, and programs.</li></ul>Leverage simplicity of a score without sacrificing detail<br /><ul><li> Intuitive weighting: assumptions based on valued activities, using business acumen
  35. 35. Empirical weighting: values derived through statistical modeling</li></li></ul><li>Automate<br /><ul><li> Don’t be a trained monkey, don’t be late with results!
  36. 36. Results have a shelf-life. To be an agile marketer you need to act and adjust quickly – dated results will put you behind the curve.
  37. 37. Outside of timely results, automation will reduce cost, introduce reliability, and provide a documented process of delivering the necessary information in the right format at the right time. </li></ul>Automate your reporting to be a repeatable, reliable process<br /><ul><li> Enable the end-user to leverage fresh results/insights to make better decisions
  38. 38. Leverage automation to save time and money; free-up your resources for analysis</li></ul>There are multiple ways that you get data into Excel using the REST URL APIs.  It is fairly straightforward process to use it to get web data and populate rows of data where you want them.  Two very simple ways to do this, of course, are: 1) use the REST URL Generator application available at https://generator.webtrends.com; and 2) to use REST URLs that are generated with the new Webtrends Analytics 9 Insight interface (http://insight.webtrends.com).<br />
  39. 39. Execution – Emerson Case Study<br /><ul><li>Situation: Emerson Network Power is a global leader in industry leading technologies and solutions for data centers, IT and telecommunications networks. Headquartered in Columbus, OH, Emerson employs over 43,000 people worldwide. In August 2010, Emerson launched a site to incorporate CMS capabilities and more in-depth analytics. While they had been running Webtrends On Demand, they had not actively been utilizing the data and reports available to them.
  40. 40. Objective: Determine a way to dynamically gather data across multiple sites and multiple profiles from Webtrends. This report will be used by multiple international owners across nine brands, each with a unique Webtrends set up and reporting structure.
  41. 41. Strategy: Talk to the end users of the scorecards, find out what is currently being reported on, what they would change, and how they use data to make decisions.
  42. 42. Tactics: Create a scorecard that contains multiple tabs for different needs / viewing levels (monthly vs. custom date ranges). Utilizing Webtrends REST URLs, and the VBA Sample Application built by Michael Love, we will be able to produce a dynamic Excel Workbook that fits the needs across all brands and marketing owners. For ease of use, we will add drop downs for profile selections and date ranges, and dynamically populate the data returned across multiple tabs and reports.</li></li></ul><li>Execution – Emerson Case Study<br />The control tab within the report fetches the data based on the user defined credentials. It allows the user to select their profile and their desired date range. This is where all of the REST URL’s are also running behind the scene (they are hidden from the users view).<br />The data tab maintains historical data for analyzing trends over time, as well as the data repository for other sheets and reports.<br />1<br />2<br />1<br />2<br />
  43. 43. Execution – Emerson Case Study<br />The Monthly Scorecard tab contains key metrics that provide a high level overview of the health of the sites. It offers drop downs for easy month over month comparisons and is updated based on current selections, including all charts, tables, and graphs.<br />Key Metrics / Conversion Points<br />
  44. 44. Execution – Emerson Case Study<br />The Custom Date Range Scorecard tab contains more detailed metrics that drill deeper into the data based on the Custom Date range selected.<br />Suggested Metrics:Search Engines, Search Phrases, Regions, Referring Domains, Top Pages<br />
  45. 45. Execution – Emerson Case Study<br />Upon delivering the scorecards to each stakeholder, feedback has been outstanding. It has been flexible and easy to use, but more importantly Emerson brand owners have been able to accomplish some key tweaks to their site / media strategy based on the data they are able to retrieve.<br />These impacts included the following:<br /><ul><li>Quickly quantified campaign success
  46. 46. Made content changes based on the top pages report
  47. 47. Modified keyword purchases
  48. 48. Identified an error on the site by seeing drastic drops in traffic and were able to make the correction and get the site back up and running</li></ul>A copy of this dashboard can be found at the URL below:<br />http://winstage.vml.com/EngageConference/Dashboard/Engage_Conference_021611.xlsm<br />
  49. 49. Execution – Agricultural Company Case Study<br /><ul><li>Situation: With offices around the world; this company produces, markets and sells hybrid seed corn in nearly 70 countries worldwide. They had been running old versions of Webtrends Software for several years, without SDC. In November of 2010, they launched a new site and with that updated their Webtrends version and incorporated SDC, so now they had accurate data but didn’t know how to consume it.
  50. 50. Objective: Create a dashboard to be used by the Marketing Manager that is visual and appealing to someone who does not have a data background. The report needs to be updatable on the fly, but can’t utilize REST URLs because the client doesn’t have the latest software version.
  51. 51. Strategy: Start with the KPIs, then develop a “fake” dashboard that doesn’t have any API functionality. This gives the stakeholder something to respond to without doing all the excel / VBA heavy lifting. Once the format is agreed to, we will build the back end using ODBC.
  52. 52. Tactics: Create a multi tab dashboard that provides an Executive Summary for the Marketing Manager with many visuals including graphs, charts, and multiple note fields where actionable recommendations will be placed. We will then train the client on how to install the ODBC driver and how to update the dashboard themselves.</li></li></ul><li>Execution – Agricultural Company Case Study<br />The input tab within the report fetches the data based on the selected profile and date range. It utilizes an ODBC driver and VBA code to gather and update data.<br />Similar to the data tab on the Emerson Case Study, the data tab in the ODBC report maintains historical data for analyzing trends over time, as well as the data repository for other sheets and reports.<br />1<br />2<br />1<br />2<br />
  53. 53. Execution – Agricultural Company Case Study<br />The Executive Scorecard tab contains key metrics that provide a high level overview of traffic to the site. It offers many visuals for quick analysis and interpretation. <br />Key Metrics<br />
  54. 54. Execution – Pioneer Case Study<br />The Content / Sub Content Group tab takes advantage of one of the key features in Webtrends and provides a report that is high level and easy to read.<br />Suggested Metrics:Visits / Page Views / Page Views/Visit / Average Time Viewed<br />
  55. 55. Execution – Agricultural Company Case Study<br />The Monthly Topline tab is used to compare year over year trending to determine how seasonality affects traffic.<br />Key Metrics<br />
  56. 56. Execution – Agricultural Company Case Study<br />The executive team was able to put the dashboard to immediate use by realizing their visitors are 85% returning, so they have ramped up their paid media efforts to expand their consumer base. <br />Understanding they want to broaden their reach, they were, however, thrilled with the retention level of the new site, as it was built to be a tool that farmers could make their own. By identifying their location, the site will display their local weather, sales team, bills / payment information, local seed prices in their area, and many other custom tools meant to keep up with the farmers of tomorrow. The dashboard confirms their strategy is working!<br />
  57. 57. Checking off on all 9 Best Practices<br />
  58. 58. Execution – Sample Scorecards<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×