e.day 2007: Eric T. Peterson "All That Can Be Measured, All That Can Be Known"

Slide 1: WebAnalyticsDemystified WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 2: All That Can Be M easured, All That Can Be Known WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 3: “Hello, My Name Is Eric Peterson” • Founder and CEO, Web Analytics Demystified, Inc. • Author of three books • Web Analytics Demystified • Web Site Measurement H acks • T Big Book of K Performance I he ey ndicators • Founder of “Web Analytics Wednesday”, “The Web Analytics Forum at Yahoo! Groups” and the “Web 2.0 Measurement Working Group” • Writer and researcher at www.webanalyticsdemystified.com WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 4: About Web Analytics Demystified, Inc. • Web Analytics Demystified, Inc. provides strategic guidance to companies working to make the most of their existing investment in web analytics • We focus on: • S taffing is s ues • Integration and reporting strategies • The proces s of actually “doing” web analytics • We provide workshops, seminars, and in-house strategic consulting • Please call me directly at (503) 282-2601 for more information WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 5: W Analytics is Easy … eb Right? WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 6: Authors, Bloggers, and Pundits S ay It Is … WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 7: Web Analytics Vendors S ay It Is … WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 8: Even the Great Google S ays It Is … WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 9: So It M Be True! ust W Analytics is Easy! eb WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 10: Or Is It? • If web analytics is so easy, why do so many of us struggle? • To gather accurate data? • To hire experienced people? • To generate relevant reports? • To produce truly actionable analys is ? • To recommend meaningful experiments ? • To explain the data to our peers and managers ? • To use data to impact the overall vis itor experience? • W indeed? hy WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 11: Web Analytics by the Numbers • The Good F • 68% of organizations have employee-managed web analytics s trategies • 47% of web analytics practitioners report making recommendations and taking action • The Bad • 22% of organizations s till only do web analytics on an ad hoc basis • 57% of web analytics practitioners s aid that “web analytics was difficult” • The Ugly • 31% of web analytics practitioners s ay web analytics ans wers less than 50% of their questions • 50% of web analytics practitioners report having considered switching jobs recently • 54% of organizations aren’t even measuring their ROIfrom web analytics WebAnalyticsDemystified WebAnalytics Demys tifie Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 12: W Analytics is HARD! eb WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 13: It’s True, Web Analytics is Hard! • There, I said it. Now you can relax and focus on learning web analytics! • You don’t have to pretend to know what the heck you’re doing all the time … • … or that you’ll s uddenly figure web analytics out overnight • … or that there are even enough resources to help you learn! • You don’t have to pretend you have all the tools you need … • … or that the tools you do have will answer all your questions • … or that you know all the questions to ask in the first place! • It’s okay to admit that web analytics is damn difficult sometimes! WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 14: This is Coni Fadigan • Coni has been using web analytics for 2 years • S he’s used WebTrends and Google Analytics extensively at www.docseducation.com • Web analytics is hard for Coni because: • There is incons istency in the literature and few good res ources telling her where to s tart • It is difficult to know “the truth” about visitor behavior, es pecially given cookie blocking • WebAnalyticsDemystifiedanalytics is WebAnalytics Demys tifie At small companies,Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com web

Slide 15: This is Paul S mith • Paul has been working online since 1995 • He uses Omniture to measure the U.S . Army recruiting web site (www.goarmy.com) • Web analytics is hard for Paul because: • Vis itor engagement is a critical measure for Paul, but “engagement” is s till very poorly defined • U.S . government web sites don’t allow the use of persistent cookies without high- level approval • WebAnalyticsDemystified tifie WebAnalytics DemysDemystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com In non-commercial sites, web analytics

Slide 16: This is Terry Chadwick • Terry has worked in web analytics for 5 years • Currently a web analyst using Coremetrics at a very large and very well-known bank • Web analytics is hard for Terry because: • Implementation is s ues caus e data incons is tency • S taff turnover neces s itates cons tant re-training • Disparate sys tems give different answers • Web analytics is poorly unders tood by management • At large companies, web analytics is hard WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com because of politics and ever-changing

Slide 17: Hard, But Not Imposs ible • I interviewed a dozen people when I talked to Coni, Paul and Terry • Universally they struggled with web analytics, but nobody said it was impossible, and nobody was giving up on web analytics (yet) • Everyone unders tood the tactical and s trategic value of getting web analytics right • Everyone expres sed a great pass ion for the data we have at our dispos al • Everyone s ounded motivated to res olve the problems they have today • Fortunately, I have a practical solution to help you sell the idea of web analytics “done right” when you get back to the office tomorrow • Better yet, my practical s olution has a catchy and appropriate acronym! WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 18: RAM Up Your W Analytics P eb Strategy! WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 19: RAMP Up Your Web Analytics S trategy • To become successful with web analytics, you need a RAMP! • RESOURCES • ANALYSIS •M ULTIVARIATE TESTING • PROCESS WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 20: RESOURCES are Technology and People • Web analytics is not just about technology • The technology is s imply a means to an end • Anyone who tells you their technology is “eas y to use” is wrong! • There is a Web Site Optimization Ecosystem you need to cons ider • People are absolutely critical to web analytics • People manage technology • People manage expectations • People provide analys is • People make recommendations WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 21: The Web S ite Optimization Ecosystem Actions Qualitative Quantitativ e Measures WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 22: ANALYSIS is the Desired Output • Waaaaaay too much emphasis is put on generating reports in web analytics • Reports are a neces sary evil • But they are still evil • Analysis is what you need to drive your business up the RAMP • Recommendations, not data • Insights, not metrics • Often times companies struggle to deploy a web analytics model capable of producing analys is on an ongoing basis … WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 23: The “Hub and S poke” Model for Web Analytics The HUB supports the SPOKES on • an ongoing basis inside the organization The HUB helps conduct • ANALYSIS, does M ULTIVARIATE TESTING, and s upports PROCESS Analytic The HUB is usually the • s “Hub” primary interface to technology • S ome companies outsource the HUB The SPOKES are individual • departments with their own reporting res ources The SPOKES bring • RESOURCES and ome WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalytics demysstified.com www.webanalyticsdemystified.com www.webanalyticsdemys tified.com PROCESS to web analytics

Slide 24: MULTIVARIATE TESTING is Mandatory • If you’re not doing some type of testing, you’re not doing web analytics! • ANALYS IS is the des ired output, but you have to do something with that output • Test landing pages , tes t creative, tes t placements , tes t images , test everything! •T esting is not as hard as you think, and it’s not as expens ive as it us ed to be • It doesn’t matter if you’re doing multivariate testing or simple A/ B testing, as long as you’re testing! • Offermatica, Optimos t, S iteS pect • MeMetrics , Amades a, GWO • Touchclarity, Kefta WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 25: Testing Lets You Cross the Action Chasm Stage 0 Stage 1 Stage 3 Stage 4 Stage 2 Relative Number of Staffing Chasm Companies Investment Chasm Action Chasm Maturity of Analytics Use Source: JupiterResearch (8/05) WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 26: PROCESS Keeps Your RAMP Together • It turns out that the rest of your RAMP goes to heck if you’re not paying attention to the actual process of “doing” web analytics • There are dozens of processes that des cribe how companies s hould us e web analytics • Most companies do well in s ome areas and (very) poorly in others • A process-oriented approach towards web analytics: • S aves time and money • Reduces dependence on individuals • Reduces mistakes, errors and omissions • Improves internal understanding of web analytics WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 27: Example Web Analytics Processes Management Processes Operational Processes • As s ign Owners hip • Plan the Implementation • Define Bus ines s Objectives • Collect and Integrate Data • Trans late Objectives into Activities • Validate Data Collection • Es tablis h Goals • Provide Internal Training • S elect Technology Partners • Define and Des ign Reports • Hire and Allocate Res ources • Conduct Analys is • Es tablis h a Web Analytics Deployment • Run Controlled Experiments Model • Augment Data Collection • Track Web Analytics Return on Inves tment • Each process should be clearly defined, having assigned owners and planned outcomes • Bus iness proces s diagrams and checklists are recommended to keep everyone focused WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 28: Proces s Drives High-Value Us e of Analytics 50% of companies using defined processes are using web analytics to support both tactical and strategic decisions Please indicate to what extent web analytics data is integrated into your organizationís decision-making process? In total how many full-time employees or full-time equivalents (FTE) in your organization are dedicated to web analytics related projects? (n=395, U.S . only) WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 29: Proces s Drives Pos itive Web Analytics ROI 50% of companies using defined processes report a positive return from their investment in web analytics “Please indicate your organization's overall return on investment (ROI) in web analytics tools and processes in regards to your total investment.” “Which of the following statements best describes the way your organization manages web analytics processes? “ (n=391, U.S . only) WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 30: Even the Indus try Analysts Love Proces s! • Megan Burns, Forrester Research “Many analys ts s ay they still s pend the bulk of their time res ponding to fire drills that keep them from high-value activities like us er education, advanced analys is , and experimentation. To get out of cris is mode, firms need to plan and implement systems that handle everyday data needs with minimal effort, and allow them to manage new requirements and one-off data reques ts with cons is tent proces s es .” “With a process-driven approach, your Web analytics analys ts have the time to do higher level activities , advanced analys is , s upport multivariate tes ting, and other activities that deliver additional incremental value to an organization.” • John Lovett, The Aberdeen Group The next level [in web analytics ] is establishing business processes s o you can us e analytics to meas ure res ults . [Proces s ] is the best way for companies to leverage the analytics platform.” WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 31: RAMP Up to Web Analytics S uccess ! • Web analytics is hard, but having a solid RAMP will make you successful! • RESOURCES • ANALYSIS •M ULTIVARIATE TESTING • PROCESS WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 32: RAMPs Have Built in KPIs • When you get RAMP in place, you can calculate the slope • Add up the number of s ucces s es you’ve had us ing web analytics in the las t s ix months • Divide by the number of times you’ve s crewed s omething up • The res ult is the s lope of your RAMP which des cribes your web analytics velocity • S teeper slopes are better, and you increase the slope by minimizing mistakes • Did you find you have a shallow slope? Don’t despair • Trus t me here, you are not alone • Remember that I told you that W ANALYTICS IS HARD! EB WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 33: Why Do You Need a RAMP? • Web analytics is hard because most companies don’t treat site measurement as a strategic business initiative • Web analytics s hould not be a s eries of fire- drills and one-off reports … cons istency is critical to s ucces s • Competing on web analytics requires big picture thinking … big thinking leads to big ROI • A solid RAM supports consistency of P approach and big picture thinking! • If you’re not fully committed to optimizing WebAnalyticsDemystified invest in a www.webanalytics demys tified.com your online business, don’t WebAnalytics Demys tifie Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com

Slide 34: Don’t S ettle for Anything Less … • You may have a “RA” or a “RAP” or an “AMP” • To be truly successful, you need a RAMP • RESOURCES XX X • ANALYSIS •M ULTIVARIATE TESTING • PROCESS WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 35: W YOU Can Do W You hat hen RAM Up to W Analytics P eb Success? WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 36: Make Millions Through Tes ting • Overstock.com aggressively leveraged ANALYSIS and MULTIVARIATE TESTING to improve site design • Analys is finds that visitors us ing s ite s earch are better cus tomers • Analys ts recommend improving the vis ibility of s ite search • Controlled experimentation allows testing of different treatments for s ite search • One s ucces s ful experiment returns a 2% increase in vis itors us ing s ite s earch • RESULT: A six-figure lift in weekly revenue, annualized to a nearly 1% increase in total company revenue WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 37: S ave Millions through Careful Analys is • A well-known brand in very competitive market leveraged RESOURCES, ANALYSIS, and PROCESS for search engine marketing • Institutionalized effective reporting for search marketing efforts • Obs erved dramatic changes in core s earch key performance indicators • Analys ts uncovered fraudulent behavior occurring in search partner network • Further analys is revealed roughly 15% of all clicks from one engine were fraudulent • RESULT: A six-figure refund check and annualized savings of roughly 5% of a multi-million dollar search marketing budget WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 38: S ave Millions on Resources • Belo Interactive Media leveraged RESOURCES and PROCESS to support a diverse and distributed organization • Central group s tandardizes on Omniture for company-wide reporting (hub-and- s poke) • Analys ts develop cus tom das hboards and automated reporting workflow • Focus on the end-user res ults in s pecific reports for s pecific bus ines s needs • Automation of reporting allows for more time to conduct analysis • RESULT: A savings of 240 man-hours annually that can be allocated to higher-value analysis projects WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 39: Make Millions Through Good Analysis • CompUS A leveraged RESOURCES and ANALYSIS to isolate a specific group of users on the web site • Reporting s hows that “Product Compare” functionality is under-utilized • Vis itor segmentation is us ed to is olate vis itors us ing “Product Compare” functionality • Analys ts dis cover that this s egment had a 33% higher average order value • Changes made bas ed on this analys is increase traffic to “Compare” functionality by 11%, increas e purchas es by 16%, and reduce abandonment from thes e pages by 56% • RESULT: An annualized increase in revenue of $2.2 million WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 40: Make Millions Through Improved Messaging • A Fortune 500 travel company leveraged MULTIVARIATE TESTING and PROCESS to improve conversion through critical online booking process • Analys ts s peculate that prospects were not s eeing “bes t price guarantee” information • Concerns were confounded by multiple mess ages and placements throughout s ite • Controlled experimentation was us ed to tes t different mes s ages and treatments • The winning format was s hown to convert at a rate 0.6 points better than the control • RESULT: An estimated lift in online bookings of $30 million annually WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 41: Did I Say M illions? Yes, M illions! WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 42: In Clos ing … • W analytics is hard! eb • People who tell you otherwise might have an ulterior motive! • You can make web analytics strategic with a RAMP • RESOURCES • ANALYSIS • MULTIVARIATE TESTING • PROCESS • When you have a good RAMP, you can have a dramatic financial impact on your organization WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 43: Thank You For Your Valuable Time Feel free to contact me at Web Analytics Demystified: Eric T. Peterson Chief Executive Officer and Principal Cons ultant eric.peters on@webanalytics demys tified.com (503) 282-2601 Download free white papers and web analytics research: www.webanalyticsdemystified.com WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com

Slide 44: www.webanalyticsdemystified.com WebAnalyticsDemystified tifie WebAnalytics Demys Demystifie www.webanalyticsdemystified.com www.webanalyticsdemys tified.com www.webanalytics demys tified.com