e.day 2007: Eric T. Peterson "All That Can Be Measured, All That Can Be Known"


Published on

"All That Can Be Measured, All That Can Be Known"

The best thing about the online channel is that it can be measured a million different ways. The worst thing about the online channel is that can be measured a million different ways. Hundreds of thousands of companies working to optimize their investment in the Internet suffer from "analysis paralysis" -- not knowing where to start with web site measurement despite having already invested in expensive tools and brilliant people.

Join Eric T. Peterson, author of Web Analytics Demystified, as he outlines the do's and don'ts of web measurement, focusing on the critical business processes behind "doing" web analytics.

Published in: Business, Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • e.day 2007: Eric T. Peterson "All That Can Be Measured, All That Can Be Known"

    1. 1. Web Analytics Demystified Web Analytics Demystified
    2. 2. All That Can Be Measured, All That Can Be Known All That Can Be Measured, All That Can Be Known
    3. 3. “ Hello, My Name Is Eric Peterson” <ul><li>Founder and CEO, Web Analytics Demystified, Inc. </li></ul><ul><li>Author of three books </li></ul><ul><ul><li>Web Analytics Demystified </li></ul></ul><ul><ul><li>Web Site Measurement Hacks </li></ul></ul><ul><ul><li>The Big Book of Key Performance Indicators </li></ul></ul><ul><li>Founder of “Web Analytics Wednesday”, “The Web Analytics Forum at Yahoo! Groups” and the “Web 2.0 Measurement Working Group” </li></ul><ul><li>Writer and researcher at www.webanalyticsdemystified.com </li></ul>
    4. 4. About Web Analytics Demystified, Inc. <ul><li>Web Analytics Demystified, Inc. provides strategic guidance to companies working to make the most of their existing investment in web analytics </li></ul><ul><li>We focus on: </li></ul><ul><ul><li>Staffing issues </li></ul></ul><ul><ul><li>Integration and reporting strategies </li></ul></ul><ul><ul><li>The process of actually “doing” web analytics </li></ul></ul><ul><li>We provide workshops, seminars, and in-house strategic consulting </li></ul><ul><li>Please call me directly at (503) 282-2601 for more information </li></ul>
    5. 5. Web Analytics is Easy … Right? Web Analytics is Easy … Right?
    6. 6. Authors, Bloggers, and Pundits Say It Is …
    7. 7. Web Analytics Vendors Say It Is …
    8. 8. Even the Great Google Says It Is …
    9. 9. So It Must Be True! Web Analytics is Easy! So It Must Be True! Web Analytics is Easy!
    10. 10. Or Is It? <ul><li>If web analytics is so easy , why do so many of us struggle? </li></ul><ul><ul><li>To gather accurate data? </li></ul></ul><ul><ul><li>To hire experienced people? </li></ul></ul><ul><ul><li>To generate relevant reports? </li></ul></ul><ul><ul><li>To produce truly actionable analysis? </li></ul></ul><ul><ul><li>To recommend meaningful experiments? </li></ul></ul><ul><ul><li>To explain the data to our peers and managers? </li></ul></ul><ul><ul><li>To use data to impact the overall visitor experience? </li></ul></ul><ul><li>Why indeed? </li></ul>
    11. 11. Web Analytics by the Numbers <ul><li>The Good </li></ul><ul><ul><li>68% of organizations have employee-managed web analytics strategies </li></ul></ul><ul><ul><li>47% of web analytics practitioners report making recommendations and taking action </li></ul></ul><ul><li>The Bad </li></ul><ul><ul><li>22% of organizations still only do web analytics on an ad hoc basis </li></ul></ul><ul><ul><li>57% of web analytics practitioners said that “web analytics was difficult ” </li></ul></ul><ul><li>The Ugly </li></ul><ul><ul><li>31% of web analytics practitioners say web analytics answers less than 50% of their questions </li></ul></ul><ul><ul><li>50% of web analytics practitioners report having considered switching jobs recently </li></ul></ul><ul><ul><li>54% of organizations aren’t even measuring their ROI from web analytics </li></ul></ul>F
    12. 12. Web Analytics is HARD! Web Analytics is HARD!
    13. 13. It’s True, Web Analytics is Hard! <ul><li>There, I said it. Now you can relax and focus on learning web analytics! </li></ul><ul><ul><li>You don’t have to pretend to know what the heck you’re doing all the time … </li></ul></ul><ul><ul><ul><li>… or that you’ll suddenly figure web analytics out overnight </li></ul></ul></ul><ul><ul><ul><li>… or that there are even enough resources to help you learn! </li></ul></ul></ul><ul><ul><li>You don’t have to pretend you have all the tools you need … </li></ul></ul><ul><ul><ul><li>… or that the tools you do have will answer all your questions </li></ul></ul></ul><ul><ul><ul><li>… or that you know all the questions to ask in the first place! </li></ul></ul></ul><ul><li>It’s okay to admit that web analytics is damn difficult sometimes! </li></ul>
    14. 14. This is Coni Fadigan <ul><li>Coni has been using web analytics for 2 years </li></ul><ul><li>She’s used WebTrends and Google Analytics extensively at www.docseducation.com </li></ul><ul><li>Web analytics is hard for Coni because: </li></ul><ul><ul><li>There is inconsistency in the literature and few good resources telling her where to start </li></ul></ul><ul><ul><li>It is difficult to know “the truth” about visitor behavior, especially given cookie blocking </li></ul></ul><ul><li>At small companies, web analytics is hard because there is simply too much data </li></ul>
    15. 15. This is Paul Smith <ul><li>Paul has been working online since 1995 </li></ul><ul><li>He uses Omniture to measure the U.S. Army recruiting web site (www.goarmy.com) </li></ul><ul><li>Web analytics is hard for Paul because: </li></ul><ul><ul><li>Visitor engagement is a critical measure for Paul, but “engagement” is still very poorly defined </li></ul></ul><ul><ul><li>U.S. government web sites don’t allow the use of persistent cookies without high-level approval </li></ul></ul><ul><li>In non-commercial sites, web analytics is hard because “conversions” happen offline </li></ul>
    16. 16. This is Terry Chadwick <ul><li>Terry has worked in web analytics for 5 years </li></ul><ul><li>Currently a web analyst using Coremetrics at a very large and very well-known bank </li></ul><ul><li>Web analytics is hard for Terry because: </li></ul><ul><ul><li>Implementation issues cause data inconsistency </li></ul></ul><ul><ul><li>Staff turnover necessitates constant re-training </li></ul></ul><ul><ul><li>Disparate systems give different answers </li></ul></ul><ul><ul><li>Web analytics is poorly understood by management </li></ul></ul><ul><li>At large companies, web analytics is hard because of politics and ever-changing resources </li></ul>
    17. 17. Hard, But Not Impossible <ul><li>I interviewed a dozen people when I talked to Coni, Paul and Terry </li></ul><ul><li>Universally they struggled with web analytics, but nobody said it was impossible, and nobody was giving up on web analytics (yet) </li></ul><ul><ul><li>Everyone understood the tactical and strategic value of getting web analytics right </li></ul></ul><ul><ul><li>Everyone expressed a great passion for the data we have at our disposal </li></ul></ul><ul><ul><li>Everyone sounded motivated to resolve the problems they have today </li></ul></ul><ul><li>Fortunately, I have a practical solution to help you sell the idea of web analytics “done right” when you get back to the office tomorrow </li></ul><ul><ul><li>Better yet, my practical solution has a catchy and appropriate acronym! </li></ul></ul>
    18. 18. RAMP Up Your Web Analytics Strategy! RAMP Up Your Web Analytics Strategy!
    19. 19. RAMP Up Your Web Analytics Strategy <ul><li>To become successful with web analytics, you need a RAMP! </li></ul><ul><ul><li>R ESOURCES </li></ul></ul><ul><ul><li>A NALYSIS </li></ul></ul><ul><ul><li>M ULTIVARIATE TESTING </li></ul></ul><ul><ul><li>P ROCESS </li></ul></ul>
    20. 20. RESOURCES are Technology and People <ul><li>Web analytics is not just about technology </li></ul><ul><ul><li>The technology is simply a means to an end </li></ul></ul><ul><ul><li>Anyone who tells you their technology is “easy to use” is wrong! </li></ul></ul><ul><ul><li>There is a Web Site Optimization Ecosystem you need to consider </li></ul></ul><ul><li>People are absolutely critical to web analytics </li></ul><ul><ul><li>People manage technology </li></ul></ul><ul><ul><li>People manage expectations </li></ul></ul><ul><ul><li>People provide analysis </li></ul></ul><ul><ul><li>People make recommendations </li></ul></ul>
    21. 21. The Web Site Optimization Ecosystem Quantitative Qualitative Measures Actions
    22. 22. ANALYSIS is the Desired Output <ul><li>Waaaaaay too much emphasis is put on generating reports in web analytics </li></ul><ul><ul><li>Reports are a necessary evil </li></ul></ul><ul><ul><li>But they are still evil </li></ul></ul><ul><li>Analysis is what you need to drive your business up the RAMP </li></ul><ul><ul><li>Recommendations, not data </li></ul></ul><ul><ul><li>Insights, not metrics </li></ul></ul><ul><ul><li>Often times companies struggle to deploy a web analytics model capable of producing analysis on an ongoing basis … </li></ul></ul>
    23. 23. The “Hub and Spoke” Model for Web Analytics <ul><li>The HUB supports the SPOKES on an ongoing basis inside the organization </li></ul><ul><ul><li>The HUB helps conduct ANALYSIS , does MULTIVARIATE TESTING , and supports PROCESS </li></ul></ul><ul><ul><li>The HUB is usually the primary interface to technology </li></ul></ul><ul><ul><li>Some companies outsource the HUB </li></ul></ul><ul><li>The SPOKES are individual departments with their own reporting resources </li></ul><ul><ul><li>The SPOKES bring RESOURCES and some PROCESS to web analytics </li></ul></ul>Analytics “Hub”
    24. 24. MULTIVARIATE TESTING is Mandatory <ul><li>If you’re not doing some type of testing, you’re not doing web analytics! </li></ul><ul><ul><li>ANALYSIS is the desired output, but you have to do something with that output </li></ul></ul><ul><ul><li>Test landing pages, test creative, test placements, test images, test everything! </li></ul></ul><ul><ul><li>Testing is not as hard as you think , and it’s not as expensive as it used to be </li></ul></ul><ul><li>It doesn’t matter if you’re doing multivariate testing or simple A/B testing, as long as you’re testing! </li></ul><ul><ul><li>Offermatica, Optimost, SiteSpect </li></ul></ul><ul><ul><li>MeMetrics, Amadesa, GWO </li></ul></ul><ul><ul><li>Touchclarity, Kefta </li></ul></ul>
    25. 25. Testing Lets You Cross the Action Chasm Relative Number of Companies Maturity of Analytics Use Source: JupiterResearch (8/05) Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Investment Chasm Staffing Chasm Action Chasm
    26. 26. PROCESS Keeps Your RAMP Together <ul><li>It turns out that the rest of your RAMP goes to heck if you’re not paying attention to the actual process of “doing” web analytics </li></ul><ul><ul><li>There are dozens of processes that describe how companies should use web analytics </li></ul></ul><ul><ul><li>Most companies do well in some areas and (very) poorly in others </li></ul></ul><ul><li>A process-oriented approach towards web analytics: </li></ul><ul><ul><li>Saves time and money </li></ul></ul><ul><ul><li>Reduces dependence on individuals </li></ul></ul><ul><ul><li>Reduces mistakes, errors and omissions </li></ul></ul><ul><ul><li>Improves internal understanding of web analytics </li></ul></ul>
    27. 27. Example Web Analytics Processes <ul><li>Each process should be clearly defined, having assigned owners and planned outcomes </li></ul><ul><li>Business process diagrams and checklists are recommended to keep everyone focused </li></ul>Management Processes Operational Processes <ul><li>Assign Ownership </li></ul><ul><li>Define Business Objectives </li></ul><ul><li>Translate Objectives into Activities </li></ul><ul><li>Establish Goals </li></ul><ul><li>Select Technology Partners </li></ul><ul><li>Hire and Allocate Resources </li></ul><ul><li>Establish a Web Analytics Deployment Model </li></ul><ul><li>Track Web Analytics Return on Investment </li></ul><ul><li>Plan the Implementation </li></ul><ul><li>Collect and Integrate Data </li></ul><ul><li>Validate Data Collection </li></ul><ul><li>Provide Internal Training </li></ul><ul><li>Define and Design Reports </li></ul><ul><li>Conduct Analysis </li></ul><ul><li>Run Controlled Experiments </li></ul><ul><li>Augment Data Collection </li></ul>
    28. 28. Process Drives High-Value Use of Analytics Please indicate to what extent web analytics data is integrated into your organizationís decision-making process? In total how many full-time employees or full-time equivalents (FTE) in your organization are dedicated to web analytics related projects? (n=395, U.S. only) 50% of companies using defined processes are using web analytics to support both tactical and strategic decisions
    29. 29. Process Drives Positive Web Analytics ROI “ Please indicate your organization's overall return on investment (ROI) in web analytics tools and processes in regards to your total investment.” “Which of the following statements best describes the way your organization manages web analytics processes? “ (n=391, U.S. only) 50% of companies using defined processes report a positive return from their investment in web analytics
    30. 30. Even the Industry Analysts Love Process! <ul><li>Megan Burns, Forrester Research </li></ul><ul><li>“ Many analysts say they still spend the bulk of their time responding to fire drills that keep them from high-value activities like user education, advanced analysis, and experimentation. To get out of crisis mode, firms need to plan and implement systems that handle everyday data needs with minimal effort, and allow them to manage new requirements and one-off data requests with consistent processes.” </li></ul><ul><li>“ With a process-driven approach, your Web analytics analysts have the time to do higher level activities, advanced analysis, support multivariate testing, and other activities that deliver additional incremental value to an organization.” </li></ul><ul><li>John Lovett, The Aberdeen Group </li></ul><ul><li>The next level [in web analytics] is establishing business processes so you can use analytics to measure results. [Process] is the best way for companies to leverage the analytics platform.” </li></ul>
    31. 31. RAMP Up to Web Analytics Success! <ul><li>Web analytics is hard, but having a solid RAMP will make you successful! </li></ul><ul><ul><li>R ESOURCES </li></ul></ul><ul><ul><li>A NALYSIS </li></ul></ul><ul><ul><li>M ULTIVARIATE TESTING </li></ul></ul><ul><ul><li>P ROCESS </li></ul></ul>
    32. 32. RAMPs Have Built in KPIs <ul><li>When you get RAMP in place, you can calculate the slope </li></ul><ul><ul><li>Add up the number of successes you’ve had using web analytics in the last six months </li></ul></ul><ul><ul><li>Divide by the number of times you’ve screwed something up </li></ul></ul><ul><ul><li>The result is the slope of your RAMP which describes your web analytics velocity </li></ul></ul><ul><li>Steeper slopes are better, and you increase the slope by minimizing mistakes </li></ul><ul><li>Did you find you have a shallow slope? Don’t despair </li></ul><ul><ul><li>Trust me here, you are not alone </li></ul></ul><ul><ul><li>Remember that I told you that WEB ANALYTICS IS HARD! </li></ul></ul>
    33. 33. Why Do You Need a RAMP? <ul><li>Web analytics is hard because most companies don’t treat site measurement as a strategic business initiative </li></ul><ul><ul><li>Web analytics should not be a series of fire-drills and one-off reports … consistency is critical to success </li></ul></ul><ul><ul><li>Competing on web analytics requires big picture thinking … big thinking leads to big ROI </li></ul></ul><ul><ul><li>A solid RAMP supports consistency of approach and big picture thinking! </li></ul></ul><ul><li>If you’re not fully committed to optimizing your online business, don’t invest in a RAMP </li></ul>
    34. 34. <ul><li>You may have a “RA” or a “RAP” or an “AMP” </li></ul><ul><li>To be truly successful, you need a RAMP </li></ul>Don’t Settle for Anything Less … <ul><ul><li>R ESOURCES </li></ul></ul><ul><ul><li>A NALYSIS </li></ul></ul><ul><ul><li>M ULTIVARIATE TESTING </li></ul></ul><ul><ul><li>P ROCESS </li></ul></ul>X X X
    35. 35. What YOU Can Do When You RAMP Up to Web Analytics Success? What YOU Can Do When You RAMP Up to Web Analytics Success?
    36. 36. Make Millions Through Testing <ul><li>Overstock.com aggressively leveraged A NALYSIS and M ULTIVARIATE TESTING to improve site design </li></ul><ul><ul><li>Analysis finds that visitors using site search are better customers </li></ul></ul><ul><ul><li>Analysts recommend improving the visibility of site search </li></ul></ul><ul><ul><li>Controlled experimentation allows testing of different treatments for site search </li></ul></ul><ul><ul><li>One successful experiment returns a 2% increase in visitors using site search </li></ul></ul><ul><li>RESULT: A six-figure lift in weekly revenue, annualized to a nearly 1% increase in total company revenue </li></ul>
    37. 37. Save Millions through Careful Analysis <ul><li>A well-known brand in very competitive market leveraged R ESOURCES, A NALYSIS, and P ROCESS for search engine marketing </li></ul><ul><ul><li>Institutionalized effective reporting for search marketing efforts </li></ul></ul><ul><ul><li>Observed dramatic changes in core search key performance indicators </li></ul></ul><ul><ul><li>Analysts uncovered fraudulent behavior occurring in search partner network </li></ul></ul><ul><ul><li>Further analysis revealed roughly 15% of all clicks from one engine were fraudulent </li></ul></ul><ul><li>RESULT: A six-figure refund check and annualized savings of roughly 5% of a multi-million dollar search marketing budget </li></ul>
    38. 38. Save Millions on Resources <ul><li>Belo Interactive Media leveraged R ESOURCES and P ROCESS to support a diverse and distributed organization </li></ul><ul><ul><li>Central group standardizes on Omniture for company-wide reporting (hub-and-spoke) </li></ul></ul><ul><ul><li>Analysts develop custom dashboards and automated reporting workflow </li></ul></ul><ul><ul><li>Focus on the end-user results in specific reports for specific business needs </li></ul></ul><ul><ul><li>Automation of reporting allows for more time to conduct analysis </li></ul></ul><ul><li>RESULT: A savings of 240 man-hours annually that can be allocated to higher-value analysis projects </li></ul>
    39. 39. Make Millions Through Good Analysis <ul><li>CompUSA leveraged R ESOURCES and A NALYSIS to isolate a specific group of users on the web site </li></ul><ul><ul><li>Reporting shows that “Product Compare” functionality is under-utilized </li></ul></ul><ul><ul><li>Visitor segmentation is used to isolate visitors using “Product Compare” functionality </li></ul></ul><ul><ul><li>Analysts discover that this segment had a 33% higher average order value </li></ul></ul><ul><ul><li>Changes made based on this analysis increase traffic to “Compare” functionality by 11%, increase purchases by 16%, and reduce abandonment from these pages by 56% </li></ul></ul><ul><li>RESULT: An annualized increase in revenue of $2.2 million </li></ul>
    40. 40. Make Millions Through Improved Messaging <ul><li>A Fortune 500 travel company leveraged M ULTIVARIATE TESTING and P ROCESS to improve conversion through critical online booking process </li></ul><ul><ul><li>Analysts speculate that prospects were not seeing “best price guarantee” information </li></ul></ul><ul><ul><li>Concerns were confounded by multiple messages and placements throughout site </li></ul></ul><ul><ul><li>Controlled experimentation was used to test different messages and treatments </li></ul></ul><ul><ul><li>The winning format was shown to convert at a rate 0.6 points better than the control </li></ul></ul><ul><li>RESULT: An estimated lift in online bookings of $30 million annually </li></ul>
    41. 41. Did I Say Millions? Yes, Millions! Did I Say Millions? Yes, Millions!
    42. 42. In Closing … <ul><li>Web analytics is hard! </li></ul><ul><ul><li>People who tell you otherwise might have an ulterior motive! </li></ul></ul><ul><li>You can make web analytics strategic with a RAMP </li></ul><ul><ul><li>R ESOURCES </li></ul></ul><ul><ul><li>A NALYSIS </li></ul></ul><ul><ul><li>M ULTIVARIATE TESTING </li></ul></ul><ul><ul><li>P ROCESS </li></ul></ul><ul><li>When you have a good RAMP, you can have a dramatic financial impact on your organization </li></ul>
    43. 43. Thank You For Your Valuable Time <ul><li>Feel free to contact me at Web Analytics Demystified: </li></ul><ul><ul><li>Eric T. Peterson </li></ul></ul><ul><ul><li>Chief Executive Officer and Principal Consultant </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>(503) 282-2601 </li></ul></ul><ul><li>Download free white papers and web analytics research: </li></ul><ul><ul><li>www.webanalyticsdemystified.com </li></ul></ul>
    44. 44. www.webanalyticsdemystified.com www.webanalyticsdemystified.com