Web Site Measurement Hacks Sample

Web Site Measurement Hacks™ by Eric T. Peterson Copyright © 2005 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly Media, Inc. books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (safari.oreilly.com). For more information, contact our corporate/institutional sales department: (800) 998-9938 or corporate@oreilly.com. Andrew Odewahn Jamie Peppard Editors: Production Editor: Mary T. O’Brien Ellie Volckhausen Cover Designer: Rael Dornfest Series Editor: David Futato Interior Designer: Dale Dougherty Executive Editor: Printing History: August 2005: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. The Hacks series designations, Web Site Measurement Hacks, the image of a combination square, and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. Small print: The technologies discussed in this publication, the limitations on these technologies that technology and content owners seek to impose, and the laws actually limiting the use of these technologies are constantly changing. Thus, some of the hacks described in this publication may not work, may cause unintended harm to systems on which they are used, or may not be consistent with applicable user agreements. Your use of these hacks is at your own risk, and O’Reilly Media, Inc. disclaims responsibility for any damage or expense resulting from their use. In any event, you should take care that your use of these hacks does not violate any applicable laws, including copyright laws. TM This book uses RepKover™ a durable and flexible lay-flat binding. , ISBN: 0-596-00988-7 [M]

Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Credits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Chapter 1. Web Measurement Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1. Talk the Talk 4 2. Best Practices for Web Measurement 10 3. Select the Right Vendor 13 4. Staff for Web Measurement Success 17 5. Get to Know Your Visitors 20 6. Understand Common Data Sources 22 7. Understand Visitor Intent 25 8. Know When to Use Packet Sniffing 27 9. Write a Useful Web Measurement Request for Proposal (RFP) 32 10. Find a Free or Cheap Web Measurement Solution 33 11. Use Analog to Process Logfiles 38 12. Build Your Own Web Measurement Application: An Overview and Data Collection 40 13. Build Your Own RSS Tracking Application: An Overview and Data Collection 45 Chapter 2. Implementation and Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 14. Optimize the Implementation Process 52 15. Improve Data Accuracy with Cookies 56 16. Know When to Use First-Party Cookies 59 iii

17. Alternatives to Cookies 62 18. Use Macromedia Flash Local Shared Objects Instead of Cookies 65 19. Fine-Tune Your Data Collection 69 20. Define Useful Page Names and Content Groups 73 21. Understand Where Data Gets Lost 75 22. Deconstruct Web Server Logfiles 79 23. Exclude Robots and Spiders from Your Analysis 83 24. Bust the Cache for Accuracy 87 25. Use Query Strings Effectively 91 26. Web Measurement and Visitor Privacy 95 27. Establish a P3P Privacy Policy 100 28. Deconstruct JavaScript Page Tags 106 29. Understand Web Bugs 111 30. Hack the JavaScript Document Object Model 114 31. Use Custom Variables Wisely 118 32. Best Practices for Data Integration 122 33. Measure Your Intranet or Extranet 126 34. Measure Your Mistakes 129 35. Build Your Own Web Measurement Application: The Core Code 134 36. Build Your Own RSS Tracking Application: The Core Code and Reporting 138 Chapter 3. Online Marketing Measurement . . . . . . . . . . . . . . . . . . . . . . . . . 150 37. Understand Marketing Terminology 151 38. Identify Your Business Objectives 156 39. Define Conversion Events 158 40. Measure Banner Advertising 163 41. Measure Email Marketing 167 42. Measure Paid Search Engine Marketing 173 43. Measure Organic Search 177 44. Contrast Paid Keywords Versus Actual Search Queries 180 45. Measure Affiliate Marketing 184 46. Use Unique Landing Pages 188 47. Measure Content Syndicated via RSS 191 48. Segment Visitors to Understand Specific Group Activity 195 iv | Contents

49. Measure Conversion Through Multiple Goals 199 50. Leverage Referring Domains and URLs 203 51. Calculate Click-to-Visit Drop-off 205 52. Create Visitor Loyalty Segments 209 53. Build Your Own Web Measurement Application: Marketing Data 213 Chapter 4. Measuring Web Site Usability . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 54. Measure the Value of Pages and Clicks 219 55. Measuring Clicks the Old-Fashioned Way 222 56. Use Language to Drive Action 223 57. Deconstruct Time Spent on Site 227 58. Use the Entry, Exit, and Single-Access Page Report 231 59. Measure Multi-Step Processes 237 60. Measure Usability in the Checkout Process 240 61. Measure “Internal Campaigns” 243 62. Use Browser Overlays 245 63. Run Your Own Split-Path Tests 248 64. Measure Internal Searches 253 65. Take Advantage of “Zero Results” Internal Search Results 256 66. Effectively Measure the “Known” Visitor 260 67. Build Your Own Web Measurement Application: Usability Data 264 Chapter 5. Technographics and “Demographics” . . . . . . . . . . . . . . . . . . . . 267 68. Measure Site Performance 269 69. Measure Connection Type 271 70. Know How to Use Screen Resolution Data 276 71. Know How to Use Browser Version Information 279 72. Know if People Are Bookmarking Your Site 282 73. Measure Browser Plug-ins 285 74. Know Which Technographic Data to Ignore 287 75. Know How to Use Visitor Language Reports 290 76. Hacking into Page-Level Details for Language 292 77. Track Demographic Data Using Custom Variables and Visitor Segmentation 294 78. Track Your Geographic Visitor Distribution 301 Contents |v

79. Accurately Measure Downloads 304 80. Build Your Own Web Measurement Application: Technographic Data 308 Chapter 6. Web Measurement and the Online Retail Model . . . . . . . . . . . 311 81. Know How to Use Retail Analytics 312 82. Measure the Shopping Cart 316 83. Measure the Checkout Process 320 84. Understand Frequency and Lifetime Value 324 85. Measure Potential Customer Value Using Recency and Latency 327 86. Manage Lifetime Value Using the Visitor Segment Value Matrix 331 87. Use Cross-Sell Data to Sell More Products 334 88. Use Geographic Segmentation to Measure Offline Marketing 338 89. Measure New and Returning Customers 340 90. Build Your Own Web Measurement Application: Commerce Data 344 Chapter 7. Reporting Strategies and Key Performance Indicators . . . . . . 349 91. Distribute Reports Wisely 352 92. Know If the News Is Good 355 93. (Don’t) Benchmark Your Site 357 94. Use Key Performance Indicators 360 95. Know the Difference Between a KPI and a Measurement 364 96. Key Performance Indicators for Online Retailers 365 97. Key Performance Indicators for Advertising and Content Sites 370 98. Key Performance Indicators for Customer Support Sites 376 99. Key Performance Indicators for Business Sites (Lead Generation) 381 100. Build Your Own Web Measurement Application: Reporting 386 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391 vi | Contents

0 Preface When the Internet was first born, most of us were so delighted with the ability to share information across great distances with relative ease that we gave little thought to critical analysis of how that information was being con- sumed. With the advent of the modern browser giving way to not just information but nice-looking information, our delight only magnified. Like children in a sandbox, we built sites, added images and content, and told everyone who would listen, “Hey, you! Come to my web site. My web site is great!” At some point, somebody asked if anyone was coming. Nobody knew the answer. The tools had not been developed, nor the practices established, to understand how people were interacting with these rapidly emerging web sites. The direct mailing crowd had cut their teeth on square inch analysis and DMA zones, and the television and radio folks had their Nielsen and Sound- scan data. Physical stores had Underhill, his planograms, and spying college students. Even telesales operations had a notion of how well received their outgoing message was, based on the number of hang-ups they were getting. Web site operators had nothing more than the occasional webmaster@ email saying someone liked the site and was it OK to copy their code. Enter web site measurement. In 1993 at Honolulu Community College, an enterprising young man (Kevin “Kev” Hughes, for the record) wrote and announced getsites 1.4, a simple web server log analyzer (Figure P-1). All of the sudden, anyone with a reasonable knowledge of C and their local filesystem could finally see what pages people were looking at. It was basic at best, but it opened the floodgates for what is predicted to become a bil- lion dollar industry by 2009. xv

Figure P-1. Announcement of getsites 1.4 In 2005, web measurement applications are as important to the Internet business framework as web servers and commerce engines. Few serious businesses spend money online without having a tool in place to measure the effect of that expenditure, providing data for critical analysis of the question “Was that money well spent?” Today, companies like WebTrends, Omniture, and Visual Sciences routinely close deals worth hundreds of thousands of dollars—all so companies can understand who is coming to their sites, where they’re coming from, and what they’re viewing in an effort to understand “why.” It is those questions we hope to answer in Web Site Measurement Hacks. Why Web Site Measurement Hacks? The term hacking has a bad reputation in the press. The press uses it to refer to those who break into systems or wreaks havoc with computers as their xvi | Preface

weapon. Among people who write code, though, the term hack refers to a “quick-and-dirty” solution to a problem, or a clever way to get something done. And the term hacker is taken very much as a compliment, referring to someone as being creative, having the technical chops to get things done. The Hacks series is an attempt to reclaim the word, document the good ways people are hacking, and pass the hacker ethic of creative participation on to the uninitiated. Seeing how others approach systems and problems is often the quickest way to learn about a new technology. There are plenty of sources for purely technical information about web data—how to parse logfiles, optimize server performance, and write cool JavaScript. Unfortunately, it is usually the “why,” not the “how,” that leaves businesses hanging. Web data collection is a simple practice, as is parsing the data into relatively meaningful buckets. The hard part is the analysis— figuring out what data is important and what it means relative to the business problem at hand. Web site measurement is something software can do, enabled by a variety of data collection algorithms and parsing strategies. Web analytics is something that requires people—bright people willing to roll up their sleeves, hunker down, and answer the hard questions. The hacks in this book are designed to help you know what to do to gain insight into how people use your web site—bits and bytes of information that will help you better explore, understand, and unearth information about how people interact with their sites. Sure, there are scripts and technical tricks, but the essence of hacking in this context is analysis. This compendium of interesting ideas, built upon a foundation of relevant and important information about how the Web is measured, is designed to turn you into a sophisticated web data analyst (or at least push you in the right direction). The result is 100 hacks, over half of which have been written by some of the best and brightest minds in web measurement today, all of which will hopefully push the limits of your understanding of web measurement, give you ideas about how better to answer the intangible “why,” and, most of all, encourage you to “hack” into your web measurement data. How This Book Is Organized You can read this book from cover to cover if you like, but each hack stands on its own, so feel free to browse and jump to the different sections that interest you most. If there’s a prerequisite you need to know about, a cross- reference will guide you to the right hack. As you can imagine, there is more involved in web measurement and analysis than we could possibly cover in 100 hacks. Each of the four dominant business models (retail, advertising, support, and lead generation) has Preface | xvii

enough subtly and complexity in how it should be measured to merit a book of its own. Still, the goal in Web Site Measurement Hacks is to get your gears turning and mind humming thinking about the most common problems companies encounter, regardless of business model. To this end, the book is broken into seven chapters: Chapter 1, Web Measurement Basics In Chapter 1, we’ll tackle the most important aspects of web measurement, especially if you’re new to the subject, including the languages used and technologies deployed, then take a look at the vendor selection process. Chapter 2, Implementation and Setup This chapter is a walk through the litany of things you need to be thinking about when you’re implementing a measurement application for your site. We cover the differences between common data sources, integration of commerce and custom data, privacy policies, and the impact that robots and spiders can have on your analysis. Chapter 3, Online Marketing Measurement The number one thing that companies do with web measurement applications is collect data that will help them justify their marketing investment. Whether you buy banner ads, send email, bid on search keywords or advertising for your site in the offline world, this collection of hacks will get you focused like a laser beam. Chapter 4, Measuring Web Site Usability More than anything, site owners want to believe their creations are easy to use and easy to understand. Unfortunately, this is rarely the case. Fortunately, web measurement tools provide a plethora of data about usability, allowing site owners to iteratively improve the overall visitor experience (hopefully for the better). Chapter 5, Technographics and “Demographics” It wouldn’t be an O’Reilly book without some geeky stuff about the ugly underbelly of the Internet, would it? Chapter 5 explores how web measurement applications can be leveraged to improve your site’s design and your internal testing and refinement strategies. Chapter 6, Web Measurement and the Online Retail Model Given the fact that there are four equally valuable business models online, how do we justify devoting an entire chapter to online retail? Simple, online retailers spend a great deal of money on web measurement, more than the other three business models combined by some estimates. This chapter deals with a dozen or so of the most common measurement needs for online retailers, including shopping carts, checkout processes, and the lifetime value of a customer. xviii | Preface

Chapter 7, Reporting Strategies and Key Performance Indicators Many vendors would have you believe that the interface they provide into the data is the only thing you’ll need to be successful. They’re wrong. Extensive interviews and experience tell us that most companies are successful with web measurement data when it’s presented in a format they’re comfortable with. In this chapter, we present key performance indicators and discuss how they can be used to improve the likelihood of adoption and action for web data. About the Use of Screenshots and Vendor Information in This Book By some estimates, there are well over 100 vendors providing web measurement tools plus nearly as many free solutions—far too many to adequately treat in a single book. The author and editor of this book have worked dili- gently to be as fair as possible in our coverage of the vendor landscape and have made every effort to distribute the inclusion of screenshots and examples as equitably as possible. That said, nobody is perfect, and you cannot please all of the people all of the time. Inevitably, some vendors’ work will be represented more frequently throughout this book. Specifically, at the time this book was being written, the author had demonstration access to applications provided by Omniture, WebSideStory, and Visual Sciences. Because of this, these vendors may appear more frequently throughout the book than, say, Urchin, Click- Tracks, or Sane Solutions. Neither slight nor preference was intended. I can assure you, it was only laziness on the part of the author that prevented each and every vendor from being represented with the exact same number of screenshots, contributed hacks, and mentions throughout the book. Conventions Used in This Book The following is a list of the typographical conventions used in this book: Italics Indicates URLs, filenames, filename extensions, and directory/folder names. For example, a path in the filesystem appears as /Developer/Applications. Constant width Used to show code examples, the contents of files, console output, as well as the names of modules, variables, commands, and other code excerpts. Constant width bold Used to highlight portions of code, typically new additions to old code. Preface | xix

Constant width italic Used in code examples and tables to show sample text to be replaced with your own values. Color The second color is used to indicate a cross-reference within the text. You should pay special attention to notes set apart from the text with the following icons: This is a tip, suggestion, or general note. It contains useful supplementary information about the topic at hand. This is a warning or note of caution, often indicating that your money or your privacy might be at risk. The thermometer icons, found next to each hack, indicate the relative complexity of the hack: beginner moderate expert Using Code Examples This book is here to help you get your job done. In general, you may use the code in this book in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD- ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution includes the title, author, publisher, and ISBN. For example: “Web Site Measurement Hacks by Eric T. Peterson. Copyright 2005 O’Reilly Media, Inc., 0-596-00988-7.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. xx | Preface

How to Contact Us We have tested and verified the information in this book to the best of our ability, but you may find that features have changed (or even that we have made mistakes!). As a reader of this book, you can help us to improve future editions by sending us your feedback. Please let us know about any errors, inaccuracies, bugs, misleading or confusing statements, and typos that you find anywhere in this book. Please also let us know what we can do to make this book more useful to you. We take your comments seriously and will try to incorporate reasonable suggestions into future editions. You can write to us at: O’Reilly Media, Inc. 1005 Gravenstein Hwy N. Sebastopol, CA 95472 (800) 998-9938 (in the U.S. or Canada) (707) 829-0515 (international/local) (707) 829-0104 (fax) To ask technical questions or to comment on the book, send email to: bookquestions@oreilly.com The web site for Web Site Measurement Hacks lists examples, errata, and plans for future editions. You can find this page at: http://www.oreilly.com/catalog/webmeasurehks For more information about this book and others, see the O’Reilly web site: http://www.oreilly.com Safari Enabled When you see a Safari® Enabled icon on the cover of your favorite technology book, that means the book is available online through the O’Reilly Network Safari Bookshelf. Safari offers a solution that’s better than e-books. It’s a virtual library that lets you easily search thousands of top tech books, cut and paste code samples, download chapters, and find quick answers when you need the most accurate, current information. Try it for free at http://safari.oreilly.com. Got a Hack? To explore Hacks books online or to contribute a hack for future titles, visit: http://hacks.oreilly.com Preface | xxi

HACK Best Practices for Web Measurement #2 Best Practices for Web Measurement HACK Hack #2 #2 To truly be successful with your online business, you need to treat web measurement as a business practice and be willing to invest time, effort, and money as necessary. Web measurement is not a silver bullet; in fact, outside the realm of law enforcement and werewolf hunting, there are no silver bullets. In order to be successful with your web measurement program, you have to treat it like any other business process through such things as customer relationship management (CRM), sales force automation (SFA), and enterprise resource planning (ERP). There needs to be an abbreviation for web measurement— for example, “WMO” for “web measurement and optimization,” which cap- tures the fact that you measure for improvement’s sake, or “SMI” for “site metrics integration,” which expresses the need to integrate your metrics with other site operation strategies. Perhaps that’s all that stands in the way of web measurement becoming widely used inside organizations: an appropriate abbreviation. The following best practices, if rigorously followed, will help you identify changes you can make that will dramatically improve your site. Identify Your Objectives Before You Begin A common mistake that many companies make is to rush out to purchase software or services before they develop sound reasons for doing so, a mistake not exclusive to web measurement. While occasionally these companies are able to back into the rationale for the purchase, a better approach is to actually sit down with those in charge and explore what you hope to gain by an investment in web measurement in advance. This is usually the best place to begin implementing a web measurement strategy: clearly identifying your site’s business objectives [Hack #38]. Some examples of clear reasons for investment include: • “We’re a retailer and our margins are very low. We want to increase the number and value of online purchases while making sure that our marketing dollars are not wasted.” • “Our customer support costs are very high. We want to optimize our support site so that more customers are likely to find answers to basic questions, decreasing the number of phone calls we get.” • “We have an online banking application that customers have com- plained about. We want to identify and fix any usability issues so more people will use this application.” Chapter 1, Web Measurement Basics 11 |

HACK #2 Best Practices for Web Measurement • “We’re a new company trying to enter a very competitive market. We need to know how best to communicate our value proposition and differentiate ourselves from bigger competitors.” When you’re clear about your overall goals, it becomes much easier to explain your needs to vendors [Hack #3], make implementation decisions (Chapter 2), develop a reporting strategy for your KPIs [Hack #91], and explain to your senior executives exactly what you’re trying to do. Make Sure You Have Executive Buy-In Web measurement works only if you’re able to make changes to your web site. Occasionally you’re going to realize that you need to make changes to your entire online strategy—something that is easier to do if you have executive buy-in. If you’re the owner of your site, great, give yourself permission. However, if you’re like most of us and you report to somebody who reports to somebody who reports to somebody, you may want to prepare yourself for an uphill battle at times. One of the best ways to avoid these internal squabbles—fights that often result in no action at all—is to ensure that management understands not only what is required but also what can be gained. To get management involved, consider having them read the following hacks in addition to Chapter 7: • Best Practices for Web Measurement [Hack #2] • Understand Marketing Terminology [Hack #37] • Define Conversion Events [Hack #39] • Identify Your Business Objectives [Hack #38] Build the Right Team One of the most important things to take away from this book is that regardless of how good the application, there is no substitute for smart people. Companies traditionally over-invest in software and under-invest in expertise to translate insight into action; web measurement is no exception. Research suggests that you’re far more likely to make good decisions based on web data if you have at least one dedicated person [Hack #4] maintaining the application and analyzing the data. The ideal situation is two people— one focused on the implementation and vendor relationship, the other charged with making sense of the data and ensuring that the rest of the organization “gets it”—both of whom report to a relatively senior person. 12 Chapter 1, Web Measurement Basics |

HACK Best Practices for Web Measurement #2 Regardless of how many people you can assign to web measurement projects, understand that zero dedicated resources will return zero valuable insights. That said, if you’re willing and able to assign resources, the continuous improvement process will help you translate effort into action. Measure and Improve: The Continuous Improvement Process Once you’ve identified your objectives, received executive buy-in, and built the right team, the real work can begin. The most reliable way to integrate web measurement into your overall business is via the continuous improvement process. Figure 1-3 illustrates an ongoing cycle of “measure, report, analyze, optimize” that takes advantage of your data collection and reporting applications; your smart people; and your desire to improve your site, customer experience, and (hopefully) top and bottom lines. Use measurement Compile data into business- application to automate legible reports using key collection of relevant performance indicators, etc. information. Measure Report Optimize Analyze Analyze data with dedicated Modify the site based on personnel enabled to insights gleaned during the instigate changes based analysis phase. on data. Figure 1-3. The continuous improvement process This framework is deceptively simple. The majority of companies have a tendency still to use web measurement as an ad hoc process: to identify a problem through some other channel (email, phone calls, or the CEO’s brother) and then look for an explanation in the metrics. This reactive use is appropriate in some situations, but should never be your only interaction with the data. The most successful companies have adopted the continuous improvement process, a proactive approach to identifying problems before the phone rings. Chapter 1, Web Measurement Basics 13 |

HACK #2 Best Practices for Web Measurement At the end of the day, following each of these best practices will help you align your organization around data, a surprisingly difficult goal. Jim Novo, measurement guru and former vice president of programming and marketing at the Home Shopping Network, has commented on many occasions that the companies who really get web measurement are companies that already “get” the use of customer data to make business decisions. Direct marketers, automobile manufacturers, book publishers, and their ilk are accustomed to mining data so they can make informed decisions. Other types of organizations are more likely to shoot from the hip and make gut-level decisions. According to Novo, the former already get it, and if you’re in this group you’ll most likely embrace the ideas in this book. If you’re in the latter group, well, keep reading. 14 Chapter 1, Web Measurement Basics |

HACK #23 Exclude Robots and Spiders from Your Analysis Exclude Robots and Spiders from Your Analysis HACK Hack #23 #23 One of the major complaints about web server logfiles is that they are often littered with activity from nonhuman user agents (“robots” and “spiders”). While they are not necessarily bad, you need to exclude robots and spiders from your “human” analysis or risk getting dramatically skewed results. Robots and spiders (also known as “crawlers” or “agents”) are computer programs that scour the Web to collect information or take measurements. There are thousands of robots and spiders in use on the Web at any time, and their numbers increase every day. Common examples include: • Search engine robots that crawl over the pages in sites on the Web and feed the information they collect to the indexes of search engines like Google, Yahoo!, or industry-specific engines that search for information such as airfares, flight schedules, or product prices. • Competitive intelligence robots that spider a site to collect competitive analysis data. For instance, your competitor may construct robots to regularly gather information from your online product catalog to understand how they should price, or to make product and price comparisons in their marketing. • Account aggregator robots that regularly collect data from online accounts (usually with the permission of the account owner) and feed that data to web-based “account consolidators.” Users of such account management sites benefit from having current information from their financial accounts, loyal program memberships (for hotel points or frequent flyer miles), or other accounts on a single site.Examples include: Everbank, Yodelee, MilePro, and MaxMiles. • Performance measurement robots that make requests of web sites to simply determine how long it will take a page on the Internet to load. Companies like Keynote and Gomez operate such robots for their clients to take measurements of their clients’ site(s) or the sites of their clients’ competitors. Your IT department or your IT vendors may use similar agents for system testing—i.e., to test that your site is up and running as intended. While great benefits are conferred when robots and spiders visit your web site, the fundamental question will always remain: are you able to distinguish requests to your site from humans from those generated by nonhuman robots and spiders? 84 Chapter 2, Implementation and Setup |

HACK Exclude Robots and Spiders from Your Analysis #23 Strategies for Limiting the Impact of Robots and Spiders The established practice in web analytics with regard to such robots is to exclude them from your analysis and reporting. The Interactive Advertising Bureau (IAB) has published the Interactive Audience Measurement and Advertising Campaign Reporting and Audit Guidelines, which include mini- mum requirements for excluding robots and spiders based on “specific iden- tification of nonhuman suspected activity” (known robots and spiders) and “pattern analysis.” For more information on the IAB and its requirements, please visit http://www.iab.net/standards/measurement.asp. To exclude robots from your analysis and reporting, provide lists of known robots to your web analytics software and configure it to filter their activity out of your web analytics data before you produce your metrics and reports. It is recommended that your robot lists are based both on IP addresses and user agents, because different user agents may use the same IP address and many robots may display the same user agent name. Identify known robots and spiders. Start with a list of known robots and spiders; such a list is likely available from your web analytics vendor. The IAB, in conjunction with ABC Interactive (ABCi), maintains a list of robots and spiders that is available to IAB members free of charge. Next, supplement the list of known robots and spiders with the names of specific user agents that have been identified by your company, such as testing agents used for site monitoring by you or your vendors. The following is a list of just a few robots that have probably visited your site: • 4anything.com LinkChecker v2.0 • Alligator 1.31 (www.nearsoftware.com) • Express WebPictures (www.express-soft.com) • DaviesBot/1.7 (www.wholeweb.net) • GomezAgent • Inktomi Search • InternetLinkAgent/3.1 • MediaCrawler-1.0 (Experimental) • Mozilla/2.0 compatible; Check&Get 1.1x (Windows 98) For more information on the IAB/ABCi list, see http://www.iab.net/standards/ spiders.asp. Be on the lookout for new robots and spiders. As a next step, establish a regular process and procedure for detecting robots that may be new to the Inter- Chapter 2, Implementation and Setup 85 |

HACK #23 Exclude Robots and Spiders from Your Analysis net or specific to your site. When you find a new robot, add it to your robots lists and have its activity filtered from your web analytics data. Save all of your old robot lists and the time range over which they were used: You’ll need to maintain versions so you can reproduce the numbers in your old reports if you ever need to. Regularly review your web server’s access logs, starting with requests for the file robots.txt, which indicates to a robot which content on your web site should be indexed. Requests for this file almost always come from a spider or robot. Don’t forget to record the user agents and IP numbers from these requests and add them to your robot lists so they are filtered from your web data. Your web measurement application should also allow you to search your web data for patterns that are common to robots. Such patterns include: • Visitors to your site that have very high numbers of page views in single sessions • Visitors to your site that have many very rapid page views or very low page view duration times • Visitors that return to your site at exact or seemingly routine times (e.g., every day at midnight) You should perform this type of pattern analysis at least once per quarter. Build and deploy a robots.txt file. The robots.txt file, which is placed within the root directory of the web site, tells spiders which files they may download and index. Most search engines will honor the robots.txt file, but there is no specific requirement that they do. The format of the robots.txt file contains two primary elements: • User-agent line • One or more Disallow lines The User-agent line is used to specify particular robots to be targeted with the use of the robots.txt file. A wildcard may be used to indicate all robots, as illustrated with the following syntax: User-agent: * The Disallow lines are used to specify particular files and/or directories that the identified user agents are not allowed to download. The format for such exclusion statements are as follows: Disallow: /homepage.asp This example instructs specified user agents not to spider the /homepage.asp file. To allow specified user agents to spider the entire web site, use the following: Disallow: 86 Chapter 2, Implementation and Setup |

HACK Exclude Robots and Spiders from Your Analysis #23 To prevent specified user agents from spidering any file within the web site, the Disallow statement would be formed as follows: Disallow: / The most common format for the robots.txt file is as follows: User-agent: * Disallow: / Modifications may be required if your site does desire search engines to index only parts of your web site or if other system visitors such as account aggregators need to access particular files/pages served from the web server. If this is the case, you should construct your robots.txt file to disallow only those parts of your site that you do not want indexed. The following is an example from a site: # robots.txt for http://www.site.com User-agent:* Disallow:/feedback Disallow:/images Disallow:/cgi-bin Disallow:/system Disallow:/inetart Disallow:/maps You can view any web site’s robots.txt file, if it has one, by requesting http:// www.domain.com/robots.txt, the standard naming convention for this file. (Replace the domain variable with the name of the site you want to check.) Remember That Some Spiders Are Good! What if you are interested in analyzing robot and spider activity rather than filtering it out? For instance, you may want to track visits from Google’s robot, Googlebot. Many web measurement application vendors offer solutions that can collect robot activity data from your web measurement data, providing the ability to analyze robot traffic for various purposes such as optimizing your pages for search engine indexing. The specifics of this procedure will vary based on your particular application, but most mature products allow you to analyze robots separately from human traffic, essentially doing the opposite of what is suggested above in this hack. It is important to know that a solely client-side data collection model (page tags) may not be able to collect all robot/ spider traffic information, because some robot/spider agents do not execute JavaScript and generally do not accept cookies. —Jim MacIntyre and Eric T. Peterson Chapter 2, Implementation and Setup 87 |

HACK Identify Your Business Objectives #38 Identify Your Business Objectives HACK Hack #38 #38 To provide real business value, you must first know what to measure and why. Fundamental to web site measurement is the question “Why do you even have a web site?” Defining your business objectives, literally, your site’s raison d’etre, is tremendously important to identifying changes you should make now and which changes you should leave for later. There’s nothing compli- cated about defining your business objectives. Usually, when you start to ask about your company’s business objectives, everybody seems to understand exactly what they are. Still, if you’re not sure, read the rest of this hack. Every Site Has Business Objectives No matter how few pages or meager your goals. If you’ve taken the time to write some HTML and FTP it up to a server, you’ve certainly done so with a goal in mind. Even the countless millions who built unattractive and poorly linked GeoCities pages had a goal in mind: for others to see their site. If you have an eBay store, a weblog, any site at all, you will undoubtedly be able to identify some type of objective. Your business objectives are always the most basic things. If you’re an online retailer your business objectives are to sell more products and support current customers. If you are growing a business selling software or services, your business objectives are to create interest and generate qualified leads. As you can see, brevity rules the day when you’re defining your business objectives, helping you craft an elevator speech (a pitch concise enough to fit in the 30 seconds or so of an elevator ride). A handful of business models and their most common objectives are listed in Table 3-1. Table 3-1. Example business objectives for common business models online Business model Business objectives Sample companies Retail Sell products Amazon Sell high-margin products WalMart Support existing customers Sears Increase revenues O’Reilly Non-packaged goods Create product awareness Toyota sales (e.g., services, Generate leads Bank of America software, durable Generate qualified leads Mercedes Benz goods) Sell products and services Comcast Customer support Decrease phone support costs Novel Increase self-service Microsoft Deliver support content PalmOne Chapter 3, Online Marketing Measurement 157 |

HACK #38 Identify Your Business Objectives Table 3-1. Example business objectives for common business models online (continued) Business model Business objectives Sample companies Advertising/content Increase advertising revenue CBSNews Increase visitor loyalty CNN.com Increase brand awareness Google Translate Business Objectives into Measurable Activities As you can see, business objectives are very high-level, 100,000 feet and above. So how can something so theoretical have any real value to web measurement? Simple. Each business objective is tied to reality by a handful of activities that can be measured via clickstream analysis. Let’s take a closer look at perhaps the Internet’s most popular business objective: sell products. How do you sell products online? Visitors click to your web site and find a product they want, they add the product to a shopping cart, and complete their transaction by checking out. Each of these individual activities (arrive, find, add, and complete) is relatively distinct and can be measured by even the most common measurement tools. Breaking the rather abstract “sell products” business objective into its constituent parts reveals what should be measured, as illustrated in Table 3-2. Table 3-2. A handful of activities that define the “sell products” business model and associated metrics Activity Metrics used to measure activity Visitors arrive at your site Campaign responses Referring URLs Search terms Entry pages Find products Path analysis and fallout reporting Product and category page views Internal search terms Add products to cart Cart start rate Product add and removal rate Complete checkout Checkout start rate Checkout completion rate Order conversion rate Buyer conversion rate Don’t worry if the measurements in Table 3-2 are foreign to you, they’ll be discussed the hacks throughout the rest of this book. 158 Chapter 3, Online Marketing Measurement |

HACK Identify Your Business Objectives #38 Once you’ve defined your business objectives and activities, you will know which reports to generate, which metrics to drill down into, which key performance indicators [Hack #94] to define and share, and even which hacks to read. It all starts with doing a good job of defining your business objectives. Chapter 3, Online Marketing Measurement 159 |

HACK Segment Visitors to Understand Specific Group Activity #47 Segment Visitors to Understand Speciﬁc Group HACK #47 Activity Hack #47 Web visitors are complex creatures, and each has slightly different behaviors and goals. Visitor segmentation is a popular strategy to differentiate these groups and develop a deeper understanding of your audience. Different visitors come to your web site for different purposes. Some come to your web site to read your content, evaluate your offerings, or make purchases. Others come looking for employment opportunities or investment information. Still others may be looking for customer support. The behavior of these distinct groups will vary a great deal, as should your goals for their membership. For example, if a web measurement report told you that only one percent of your total visitors complete an important task, you may think that your web site is failing miserably. However, if you segment your visitors, focusing only on visitors who respond to a targeted email campaign, you may find that 30 percent of these visitors complete the task. Given differences in browsing habits and ultimate goals, it certainly makes sense to leverage your measurement toolset to segment visitors in meaningful ways and create different sets of metrics for each. Fortunately, many of the top web measurement vendors offer some type of visitor segmentation tools that provide for differentiation of visitors (Figure 3-13). Examples of Visitor Segments No two web sites are likely to benefit from the exact same visitor segments; different analysts will use different criteria to examine the same behaviors, drawing different conclusions. It is likely that the segments you’re interested in will change over time as your understanding of your audience evolves. Visitor segments are typically very specific to individual businesses. That said, keep the following in mind as you brainstorm possible segments: • Your site’s varying constituencies (such as buyers, support customers, or tire kickers) • The different information you offer to each of your constituencies (such as conversion reports, KPIs, lists of pages viewed, or referring domains) • The various marketing campaigns you run to attract new visitors in each group (such as banner advertising, email, or RSS feeds) • The particular role you have as a web data analyst and the aspect of the business you’re responsible for (such as marketing, merchandising, site operations, ‘or loyalty programs) For example, as a marketing manager for a commercial web site, you may care about the visitors who are acquired via pay-per-click advertising. As a Chapter 3, Online Marketing Measurement 197 |

HACK #47 Segment Visitors to Understand Specific Group Activity Figure 3-13. Visitor segmentation product or merchandise manager for the same web site, someone else may care about the smaller slice of visitors who clicked on the pay-per-click advertisement for a specific paid keyword, and performed a local search on the web site for related merchandise, but left the site without making a purchase. Your customer service manager may care about the segment of customers who searched the self-help content but finished their visit on the “contact us” page, apparently not finding what they were looking for. General Requirements for Segmentation Visitor segmentation is entirely driven by the abilities of your web measurement application. Put another way, if your particular solution doesn’t support visitor segmentation, you can either get a new solution or not segment your visitors. Here are some general requirements that your measurement application needs in order to support to segment visitors: 198 Chapter 3, Online Marketing Measurement |

HACK Segment Visitors to Understand Specific Group Activity #47 • The ability to define a segment based on any applicable filtering criteria, such as pages or query strings viewed during visits, or the duration of visits • The ability to customize any web measurement report by restricting it to a specific visitor segment or segments. • The availability of detailed, historical web traffic data records that allow you to query historical data by slicing it into newly defined segments. The last item is often considered a “nice to have” requirement, as many web measurement solutions provide only “move forward” segmentation—the ability to track segments from the time they’re established, but not prior to that date—as opposed to ad hoc segmentation from any existing data. Ad hoc segmentation because it’s very difficult to know in advance what you’ll want to know later on. As usual, if you have any questions about your vendor’s ability to segment your visitors in meaningful ways, the best advice is to pick up the phone and give them a call. Deﬁning Good Visitor Segments The following are just a few basic examples of typical segments with hints on how you can define each segment based on the data available to you about your visitors. New versus returning visitors Perhaps the most basic, but most valuable, visitor segment, you should definitely create a segmentation report for new versus returning visitors. By taking a closer look at the differences between which pages each type of visitor is looking at, you can hopefully learn how to convert more “new” visitors into “returning” visitors. Conversion success Maybe the most frequent type of segmentation applied by web analysts is to distinguish between visitors who complete a critical action and those who do not. Depending on the mission of your web site, that action may be completing a registration form, making a purchase, or finding a support document without dialing your call center. Here you would define the segment as the slice of visitors who have completed the success action, usually measured as a view of a specific page during their visit (for example, a thank you page). Visitor acquisition source To segment by visitor acquisition source, you would define segments by creating unique landing pages [Hack #46] for each of your marketing campaigns. Any visitor who starts her visit on one of these pages is assumed to have come from the related marketing campaign and should thus be assigned to the appropriate segment. Chapter 3, Online Marketing Measurement 199 |

HACK #47 Segment Visitors to Understand Specific Group Activity Purpose of visit Without interviewing a visitor, it is not possible to know for sure what the purpose of his visit is. However, you can attempt to infer his purpose from the type of pages that the visitor is viewing or the order in which he views them. For example, you can define the segment of pro- spective customers as those visitors who view pages related to your offering. Similarly, you can define self-help visitors as those who spend time on your customer service section. Product interest or purchase A merchandise manager may wish to distinguish visitors by product interest in order to better understand how visitors research her line of products. This requires defining segments based on the products or product categories viewed or purchased during a visit. Make sure to differentiate “buyers” from “tire kickers” in this type of segmentation so that you have data to help identify why the tire kickers convert. Value of the visitor You probably want to focus some of your analysis on high-value customers to find out how they find your web site and navigate it. While the specific definition of “high value” differs greatly from site to site, in general: • As a retailer, you may care about customers whose order value exceeds a certain amount. The order value is typically captured from a URL parameter or tag that you set aside on your order confirmation page. • As a content web site owner, you may care about customers with more than five visits per week. You can track the number of repeat visits per visitor if you are using cookies or authenticated user- names to identify repeat visitors. By segmenting high-value visitors, you will be able to mine their habits in an effort to create more high-value visitors. Look for clues in their referring sources (for example, do they come from a special set of sites?), their product interests (for example, do they browse and buy a certain set of products?), and their recency and frequency of visit (for example, do they visit more frequently than lower value customers?). Tying It All Together At the end of the day, visitor segments help you better understand your visitors as distinct groups. By culling customer support visitors out, you’ll be able to generate more accurate buyer conversion rates. By removing non- customers from your support segment, you’ll be able to better understand the challenges facing your paying customers. By segmenting visitors from a 200 Chapter 3, Online Marketing Measurement |

HACK #58 Use the Entry, Exit, and Single-Access Page Report Use the Entry, Exit, and Single-Access Page ReportH HACK ack #58 #58 When you boil it down, your ability to understand visitor interaction with individual pages is one of the most important things you’ll do with your web measurement application. Knowing where visitors enter and exit your site, and which pages are least engaging, is fundamental to this knowledge. Depending on your site goals, there are a number of different metrics and reports you will want to review. It is however extremely unlikely that you won’t take a close interest in your entry pages, exit pages, and single-access pages. No online marketing program is complete without taking a close look at one or all of these page reports, as the information they provide about leakage, slippage, and stickiness in your site is absolutely invaluable. Fortu- nately, no matter what web measurement tool you are using, these three reports are part of the standard report set. Entry Pages An entry pages report displays the most commonly used pages for entering the site. This is the first page that visitors see when they come to your site. Upon reviewing this report, you may be surprised to learn that 100 percent of your site visitors don’t enter the site through the home page. In fact, they may not even see the home page at all during their visit. There are a number of reasons why people enter the site through pages other than the home page, including: • Search engine results that point to internal pages • Campaign landing pages of all types, including offline promotions • Bookmarks • URL passing among friends or colleagues or designed viral marketing efforts • False entries From this list, the final entry (“false entries”) is the one element that should be of concern to Internet marketers and is worth a deeper look. False entries are usually caused by cached pages, missing tracking (for tag-based solutions), and the technical expiration of a visit when the visitor was still engaged. Page caching. This is a more common problem when analyzing logfiles rather than tracking tags placed on the pages. If a page is not served from the server and you are relying on traditional logfiles, you won’t see that first page in the logfiles or web analytics reports. But if the second page is pulled from the web server (not cached), it would look like the second page viewed 232 Chapter 4, Measuring Web Site Usability |

HACK Use the Entry, Exit, and Single-Access Page Report #58 was the entry page to the site according to your tracking tool. You can avoid this problem by using tracking tags. Missing page tracking tags. There are unique problems when relying on tracking tags to track visitor behavior if they aren’t implemented correctly. Often, new pages are launched without the necessary tracking tags. If a site visitor enters the site on a page that is missing tracking tags, the first page he views that does have the tracking tag will show up as his entry page. Expiration of visit session. All tracking tools have a time limit when they end a visit session after no behavior. If you leave your a computer in the middle of a visit, your measurement tool will likely consider your visit over after a certain period of inactivity [Hack #1]. When you return and click a link on the site you were on before, it will consider you as starting a new visit and record you as a new entry. That can help explain why sometimes you see entries behind secure portions of the site that require login. Your tracking tool considers it a new visit, while the web site knows you are still logged in. It is important to understand entries to the site to ensure you are providing the right content and calls to action to the right people based on the top entries. You may also find that people convert on your desired behaviors (sales, leads, etc.) at higher or lower rates, depending on where they enter the site. This can help you identify some of the drivers to those conversion behaviors. Exit Pages Exit pages are the last pages people view before they leave the site. The same principles apply to “false exits,” which we described above as “false entries.” It is important to understand that all visitors to your site ultimately leave your site, and they have to leave from some page. Also, there are good places and poor places for people to exit the site, based on your overall site goals. It can be misleading to look only at the top few pages listed in the entry and exit page reports. Take a few minutes and compare the top 20 pages viewed on your site and the top 20 exit pages on your site. In many cases, the pages that receive the most traffic also record the highest number of exits. The better way to look at it is to create an exit ratio report—a comparison of page visits to page exits (Figure 4-4). You can create this report within Excel as part of your normal key performance indicators [Hack #94] and sort the pages on the site from those with the highest exit ratio to the lowest. When looking at the exit ratios, it is helpful to break the pages into different categories. The most common categories include: Chapter 4, Measuring Web Site Usability | 233

HACK #58 Use the Entry, Exit, and Single-Access Page Report Figure 4-4. Exit page ratio report Home page From an exit standpoint, your home page should be considered unique from all other pages on the site. Typically, a home page exit rate of less than 20 percent is desirable, indicating that the majority of visitors are clicking deeper into the site. Be aware that many sites that offer private login sections drop people to the home page when they select “log out,” driving up the home page exit rate significantly. Destination pages These content pages provide the information that users seek. Transition pages The only purpose of these pages is to provide options for people who are looking for deeper content on the site. Consider a banking site that has a main product page that lists all the products—it does not really provide any information, but it helps direct people to the specific product pages. Within each of these, we are looking for: Natural exit pages This is where we expect people to leave. It may be the confirmation page of a shopping cart or the page with a completed lead conversion form. “Unnatural” exit pages These are key conversion or transitional pages where you don’t want to lose visitors. 234 Chapter 4, Measuring Web Site Usability |

HACK Use the Entry, Exit, and Single-Access Page Report #58 Look through the important conversion pages recording high exit ratios and for pages that influence the most site visitors. Focus on improving these pages to reduce the exit ratio. You can strengthen calls to action, improve navigation, and cross sell other content on the site. Single-Access Pages Single-access pages are really just a combination of an entry and an exit without viewing any other pages on the site. A page is recorded as a single- access page if a visitor comes to a site, views only one page, and then exits. It is recorded as an entry to the site, an exit from the site, and a single-access page visit. These types of pages are almost always recorded by your measurement application and reported in a “single-access page” report (Figure 4-5). Figure 4-5. Single-access pages report These can be a real problem: you have done all the things you need to do to drive people to your site: they come, view one page, and then leave. It is difficult to come up with a scenario when viewing a single page on a site can be considered a positive for the site owner—no matter what your business model. When calculating exit ratios, look at what percentage are single- access pages. You can work to improve these pages the same way you address pages with high exit ratios. You need to move people from those pages recording high single page visits to other pages on your site. We aren’t going to eliminate single-access pages all together—that is not our intent—rather, we are just trying to drive more people further into the site so we have a better chance of convincing them to convert. Chapter 4, Measuring Web Site Usability | 235

HACK #58 Use the Entry, Exit, and Single-Access Page Report Unfortunately, many single-access page visits come through search engines, campaigns, and pay-per-click campaigns. And guess what—you may pay top dollar for those campaign visits that click over once and instantly bail. This is another good reason to track campaign ROI through to conversion rather than just clicks. It is easy to see how these three metrics are related, and the importance of understanding them on their own as well as together. Depending on your site and overall site goals, there are a number of ways you can use this information to improve site performance. Using All Three Reports Together While each report is powerful on its own, two very important ratios can be generated on a per-page basis that form useful key performance indicators. Page “stickiness.” .It is tremendously important to your marketing initia- tives that visitors see more than just a single page when they arrive at your site. While looking at your single-access page report is valuable, as is looking at your entry page report, the concept of “stickiness” of a page is another useful way to examine the likelihood that a page is a strong positive contrib- utor to your marketing programs.Calculate page “stickiness” on a page-by- page basis using the following: 1.00 – (Single-Access Page Visits / Entry Page Visits) For example, you would determine how many times your home page was the entry page for a visit and the home page was the only page in a visit, and use the above formula to calculate your home page stickiness. Needless to say, the more sticky the page, the more valuable it is. Any landing pages that have a stickiness of less than 40 percent should be closely examined for usability and performance issues. Ratio of page entries to exits. The ratio of entries to exits for a page provides you a rough proxy for the popularity and value of a page. The calculation will yield a ratio between 0.0 and, well, a very large number, depending on the page in question. The lower the number, the less popular the page (e.g., people leave from it more frequently than they arrive at it); the higher the number, the more popular the page. This calculation should be used in the context of the page—ask yourself “Does that make sense, given what I hope that visitors do on that page?” If you’re looking at your shopping cart “Thank You” page, a very low number would be appropriate (for example, most people will exit the site after mak- 236 Chapter 4, Measuring Web Site Usability |

HACK Use the Entry, Exit, and Single-Access Page Report #58 ing a purchase, and nobody should be entering at that page). Conversely, the higher the better for your home page, although you rarely see exception- ally large numbers on your most generic of all landing pages. While few web measurement applications calculate page stickiness and the ratio of page entries to exits, these views of your page activity are too important to ignore. You should add each of these ratios to your key performance indicators and monitor them frequently. —Jason Burby and Eric T. Peterson Chapter 4, Measuring Web Site Usability | 237

HACK #63 Run Your Own Split-Path Tests Run Your Own Split-Path Tests HACK Hack #63 #63 While there are a number of vendors providing split-path testing tools, relatively simple code will let you run your own tests. While there are a handful of vendors that provide solutions and services for split-path testing—the practice of randomly showing visitors alternative pages or content to improve conversion—most charge between $25,000 and $100,000 annually for their services. However, if you have a reasonable amount of control over your web application platform, it’s not unreasonable to simply build the ability to distribute and track visitors right into your existing site. Regardless of how you choose to implement split-path testing, the essence of a good strategy is as follows: 1. Check for test participation. 2. Assign the visitor to a test or control group. 3. Tag the visitor. 4. Redirect test subjects to the appropriate page. 5. Monitor for completion of goals. Perhaps the most important is step 5: monitoring for the completion of goals. Keep in mind that split-path testing is a lightweight adaptation of the scientific method—you have a control group and one or more tests, and you’re looking for the differences between the two sets. In the case of a web site, the differences should be measured by an increase in leads collected, sales generated, pages viewed, etc., depending on your particular business objectives [Hack #38]. The Code The following code is written in VBScript for Microsoft’s Active Server Pages, although it could be quite easily adapted to PHP, Perl, or Java. You should save the following code as split-path_testing.inc and plan to include it in your header files (ideally, via a server-side include). <% '****************************************************************** ' Define the tests to be performed. ' Tests are stored in an array. Be sure to dimension the ' array to the number of tests minus 1!@ '******************************************************************* Dim Test(1) ' SET TO MAXIMUM NUMBER OF TESTS MINUS ONE Test(0) = \"Home_Page_Test,index.asp,index_test.asp,12/1/2004,12/31/2004\" Test(1) = \"Buy_Now_Test,buy_now.asp,buy_now_test.asp,12/1/2004,12/11/2004\" Dim SplitArray Dim TestName(20), TestFile(20), DefaultFile(20), TestStart(20), TestEnd(20) 250 Chapter 4, Measuring Web Site Usability |

HACK Run Your Own Split-Path Tests #63 For i=0 to Ubound(Test) SplitArray = split(Test(i), \",\") TestName(i) = SplitArray(0) DefaultFile(i) = SplitArray(1) TestFile(i) = SplitArray(2) TestStart(i) = SplitArray(3) TestEnd(i) = SplitArray(4) Next ' Define a function called RandomNumber that will be used ' to assign visitors to the test or control group Function RandomNumber(intHighestNumber) Randomize RandomNumber = Int(Rnd * intHighestNumber) + 1 End Function ' The bulk of the code first sets a few variables including the current date and the name of the || script currently being loaded in the visitor's browser (ScriptName): Response.Buffer = True ScriptName = Request.ServerVariables(\"SCRIPT_NAME\") ThisDate = FormatDateTime(now( ), vbShortDate) ' The rest of the code simply iterates through the tests already loaded into arrays to see if the page is one included in a test, if the visitor is already in a test group and if not, which ' test group they should be assigned to. As soon as the test assignment is made, the visitor is ' then redirected (if they've been determined to be part of a test) or nothing happens and the ' rest of the page is loaded For i = 0 to Ubound(Test) if InStr(ScriptName, DefaultFile(i)) then if (CDate(ThisDate) >= CDate(TestStart(i)) AND CDate(ThisDate) =< CDate(TestEnd(i))) then if IsDate(Request.Cookies(\"TestCookie\")(TestName(i))) then Response.Redirect(\"./\" & TestFile(i) & \"?TestGroup=\" & TestName(i)) elseif Request.Cookies(\"TestCookie\")(TestName(i)) <> \"CONTROL\" then Response.Cookies(\"TestCookie\").Domain = \"www.webanalyticsdemystified.com\" Response.Cookies(\"TestCookie\").Expires = now( ) + 365 ' Here is where we actually assign the visitor to the test or control group. ' Note the TestGroup=[TestName] in the redirect - the critical piece to allow us ' to tag the visitor: if RandomNumber(100) > 50 then Response.Cookies(\"TestCookie\")(TestName(i)) = ThisDate Response.Redirect(\"./\" & TestFile(i) & \"?TestGroup=\" & TestName(i)) Chapter 4, Measuring Web Site Usability | 251

HACK #63 Run Your Own Split-Path Tests else Response.Cookies(\"TestCookie\")(TestName(i)) = \"CONTROL\" end if end if ' If the end date for the test has expired, clean out the cookie so that the test name can be reused if necessary: elseif CDate(ThisDate) > CDate(TestEnd(i)) then Response.Cookies(\"TestCookie\")(TestName(i)) = \"\" end if end if next %> Check for Test Participation: Pages and People You need a way to keep track of whether the viewed page is part of a test or not. To do, use a simple array to keep track of the name of the test (for your measurement application, a nice, readable name), the default and test filenames (physical script names), and the start and end dates for the test. Each element in the array holds information about an individual test. <% Dim Test(1) ' SET TO MAXIMUM NUMBER OF TESTS MINUS ONE Test(0) = \"Home_Page_Test,index.asp,index_test.asp,12/1/2004,12/31/2004\" Test(1) = \"Buy_Now_Test,buy_now.asp,buy_now_test.asp,12/1/2004,12/11/2004\" For the most part, all you really need to know is that the format for the test definitions is important: you have to have the name of the test, followed by a comma, followed by the default filename (e.g., the control group), followed by a comma, the name of the test file, comma, the start date in MM/DD/ YYYY format, comma, and then the end date in MM/DD/YYYY format. You also want to set a cookie in the visitor’s browser that lets you track his participation in your tests. The code to do this is very simple: Response.Cookies(\"TestCookie\")(TestName(i)) = ThisDate You create a cookie called TestCookie with a name/value pair where the name of the test (TestName(i)) equals today’s date (ThisDate). Then you’ll be able to check the cookie for a valid date to see if the visitor is participating in the test. if IsDate(Request.Cookies(\"TestCookie\")(TestName(i))) then Assign to a Test or Control Group Assuming the visitor is not already participating in a test, the next step is to randomly assign him to the test group (so that he will see the test pages) or the control group (so that he will see the control pages). While there are 252 Chapter 4, Measuring Web Site Usability |

HACK Run Your Own Split-Path Tests #63 many ways to do this, I like to use the simple “heads you’re in, tails you’re out” strategy: If RandomNumber(100) > 50 then Basically, using a random-number generating function in VBScript, you generate a number between 1 and 100; 1 through 50 are in the control and 51 to 100 are in the test group. Visitors are assigned to test or control groups on every page you’re testing in order to preserve random distribution throughout your site. Tag the Visitor If a visitor is going to be part of a test, you need to let your measurement application know this. The easiest way to do this is to modify the URL making the request for the test file so that either your logfile can be mined for the presence of a name/value pair like ?TestGroup=[TestName] or you can use this information to load a variable for your JavaScript page tag, e.g.: var _abTestGroup=\"TestName\"; We do this by appending to the end of the test filename, contained in the TestFile(i) array variable. Response.Redirect(\"./\" & TestFile(i) & \"?TestGroup=\" & TestName(i)) Note that if your test filename already has parameters in the query string, you’ll need to convert the “?” to an “&” for this redirection to work properly. Keep in mind that the method you use to identify the test visitor will be defined based on which data source you’re using [Hack #3] and specific to the application you’re using. Redirect Test Subjects to the Appropriate Page As long as the TestFile(i) variable is properly set, the visitor is going to be redirected along to the test page and assigned membership to the appropriate group. One thing you want to keep track of is whether the distribution of visitors is roughly 50/50, based on our simple test assignment strategy. Monitor the traffic to both the control and test pages to make sure they’re receiving roughly the same number of page views; if they’re not, something may have gone wrong with the code. Chapter 4, Measuring Web Site Usability | 253

HACK #63 Run Your Own Split-Path Tests Monitor for Completion of Goals This is the single more important thing you’ll do with this code, though it has little or nothing to do with the code itself. You need to make sure that you’re using the ?TestGroup=TestName in the query string to let your measurement application know that this visitor has to be tracked as a separate visitor segment [Hack #48] or member of an A/B test. Again, how you make this assignment really depends on which data source and type of application you’re using, but a phone call to your vendor should yield a pretty simple explanation about how to do this. Ideally, if you’re using a moderately powerful measurement application, you’ll then be able to see whether members of the test group are completing goals more frequently than those in the control group. This is important since it’s the only reason you would use the following code. Running the Hack The best way to run this hack is to make sure that split-path_testing.inc is included in your common header file using a server-side include.  If you don’t have a common header file, I recommend you create one rather than adding this code to every page on your site. This is the most efficient way to make global changes to the array that defines the tests, including ending all of the tests if you need to. Don’t forget, the most important thing I’m not explicitly showing you in the code is making sure that your measurement system knows about the tests. Get- ting test subjects into appropriate visitor segments and tracking those segments through the completion of goals is far and away the most important piece in split-path testing. Be sure and consult with your application vendor when you’re setting this up to capture as much useful information as possible. Hacking the Hack When you get really good at split-path testing, keep in mind that the code is already set up to let you test multiple pages at once. Because you’re building an array of tests, you can add as many as you’d like and really work hard to optimize your web site. The most important thing to keep in mind is that because arrays start at 0 rather than 1, you need to redimension the Test array to one less than the total number of tests you’re running or you’ll get a nasty ASP error. For example, if you had 100 different tests, you would redimension the Test array to 99, and then add additional elements for each new test: 254 Chapter 4, Measuring Web Site Usability |

HACK #74 Know Which Technographic Data to Ignore Know Which Technographic Data to Ignore HACK Hack #74 #74 Not all technical data is as useful as it looks at first glance. Knowing what to pay attention to and what to ignore can save you time and prevent frustration. As you’ve certainly surmised by now in reading this book, web measurement applications provide a wealth of information about your visitors, most of it good! While the vendors certainly mean well, often they provide information because they can—not because there is a great business reason for doing so. Unfortunately, not all the available information is useful, especially when you’re talking about technographic reports (Figure 5-13). Figure 5-13. Representative sample of technographic data Since most vendors provide the same technographic data, it is worth reviewing which of this information you should use and which you should ignore. Technographic Data to Use The following are technographic data points and reports that are generally useful to web data analysts. 288 Chapter 5, Technographics and “Demographics” |

HACK Know Which Technographic Data to Ignore #74 Browser type Your web developers and quality assurance group will benefit from a complete list of visitor browser types. Use recent (last 90 days) samples to ensure QA efforts map well to current browser trends. Browser width A central concern for web developers is how much screen real estate to use, and the schism between 800x600 and 1024x768 screen resolutions. Keep a close eye on how these numbers evolve; looking for the opportunity to use more screen real estate [Hack #70] will help you make more effective design decisions. Cookies The cookies report provides a partial glimpse into the accuracy [Hack #15] of your data if you’re using a page tag or augmenting your web server logfiles with a cookie. While not the final word on cookie acceptance by your visitors, I recommend checking this report on a monthly basis to look for any large decreases in cookie acceptance. Connection type While this report is almost always built from an obscure setting available via the JavaScript DOM for Internet Explorer users, as long as your IE browser share is high, this report can help you understand whether most of your visitors are broadband or modem users, helping you refine your page design and development strategy. Technographic Data to Ignore The following are technographic data points and elements that are unlikely to provide valuable insight into your visitors, either because they’re too granular, not granular enough, or because they’re otherwise useless. Browsers While the specifics of browser type are useful from a development standpoint, watching the browser wars play out is interesting, but not useful. Browser height Simply put, if you do a good job presenting content, your visitors will scroll. Alternatively, the “fold” (bottom-most point in a browser’s initial load without any scrolling) is well-defined by the browser’s width using a standard calculation. Monitor color depth You should build well-designed web pages that use web-friendly colors. Monitor resolutions The screen width report provides relevant and useful information. Chapter 5, Technographics and “Demographics” | 289

HACK #86 Manage Lifetime Value Using the Visitor Segment Value Matrix Manage Lifetime Value Using the Visitor Segment HACK #86 Value Matrix Hack #86 Combine the measurements of current value and potential value to refine your business’s customer marketing and retention strategy. What happens if you look at both the current and potential value of visitor or customer segments at the same time? You get the four groups shown in Figure 6-5. Low potential value, High potential value, high current value low current value Grow these Keep these customers customers Low potential value, High potential value, Low current value low current value Current value Ask “should i spend Grow these money here?” customers Potential value Figure 6-5. Visitor segment value matrix (courtesy of Jim Novo) How do you create your own visitor segment value matrix? Easy: 1. Take your customer segments and rank them by potential value (recency or latency [Hack #85]), and then split them into two groups: above average and below average. 2. Take all of these potential value groups and rank them by current value (frequency or lifetime value [Hack #84]), then split them into two groups: above average and below average. You will end up with the four classifications above, each containing unique visitor segments. 3. Do an analysis like this every month so you can compare the results with your financial statements. Consider how powerful it would be to know the ranking of visitor segments based on this model. The segments in the upper-right box are the rocket fuel of the company. They are the 10 percent of the segments that create 90 percent of the profits—now, and in the future. This is where you should focus customer retention efforts [Hack #52]. The segments in the lower-left box are a 332 Chapter 6, Web Measurement and the Online Retail Model |

HACK Manage Lifetime Value Using the Visitor Segment Value Matrix #86 drag on the company; they are the result of poorly targeted customer acquisition programs, for example. You should stop spending incremental marketing or service money on these segments—don’t “fire” them, but don’t spend a bunch of money on them either. The upper-left and lower-right boxes in the matrix represent the best targets for customer value enhancement programs. This is where the majority of money is made in loyalty programs, for example. The bulk of the marketing budget should be spent in trying to move these segments toward the upper- right box. If you have created the matrix above, you have hacked the equation of lifetime value (current value plus potential value equals lifetime value). Why spend all your time trying to figure out the absolute lifetime value of a customer when a relative value is really all you need? All you need to know to allocate spending is that this segment or customer is more valuable, less valuable, or its value is changing. And then you allocate resources based on the relative value of the customers or segments. That’s not to say you shouldn’t measure lifetime value, because it’s very important. But if you are a new business or don’t have patience to measure lifetime value, relative value as determined by the visitor value segment matrix is a useful substitute. Use the Matrix to Drive Content Decisions Now, think about the fact that certain media, offers, copy, content, and products are responsible for customers being in each of the four groups above. Those in the top group can easily generate many, many times the profit of those in the bottom group for a company. If you are choosing which media, offers, copy, content, and products to offer to visitors, you are choosing how many visitors end up in each of the four groups above. As you can see, the most profitable retention program you can probably execute in the short-term is to engage in some fine-tuning on your acquisition efforts. Figure 6-6, a report using frequency and latency, will help provide an example of how to peg visitors segmented by campaign to the matrix. Item 7, the “Free Regular Shipping on Electronics Email” campaign is deliv- ering visitors who come back more often—high average frequency—and have the lowest likelihood of defection—low average latency. This campaign is generating a “rocket fuel” visitor segment when compared with all the other campaigns. Very often, when designing retention programs, people worry about the dangers of allocating marketing budgets like this—what if a segment in the lower-left box suddenly has the potential to become an upper-right box seg- Chapter 6, Web Measurement and the Online Retail Model 333 |

HACK #86 Manage Lifetime Value Using the Visitor Segment Value Matrix Figure 6-6. Use of average frequency and latency to understand how campaigns segment into the visitor value matrix ment, and you have been ignoring them? Well, in the first place, it doesn’t happen very often, and the amount of money you will waste trying to make it happen will far exceed any benefit you might get. Retention marketing techniques are all about allocating precious budgets to the highest return on investment (ROI) activities, and the ROI is more likely to be lowest in the lower-left box. So as long as you are comfortable with not driving the highest profitability possible, by all means, spend money marketing to them. Besides, this kind of model does not operate in a vacuum; there are built-in checks and balances. Because this is a ranking model, as the “status” of a segment changes, so does its place in the matrix. If a segment in the lower-left box were to show up in the next analysis in the lower-right box, you could still do something about it: this segment has newly defined potential and deserves some kind of marketing program to encourage that potential. Similarly, a segment in the “rocket fuel” box that shows up in the next analysis in the upper- left box is a best customer segment in the process of defecting and needs attention right away. Something has happened to this segment—did you change the web site? Did you change the terms of service? Whatever it is, action needs to be taken to retain this best customer segment. You can waste a ton of money trying to change the value of a customer. It is far more profitable to recognize when change is taking place and either help it accelerate, as in the case of a segment increasing in value, or slow it down, as in the case of a segment defecting or decreasing in value. These are the situations where the ROI is the highest. 334 Chapter 6, Web Measurement and the Online Retail Model |

HACK Manage Lifetime Value Using the Visitor Segment Value Matrix #86 Hacking the Hack To hack the hack, don’t just report on the customer value matrix, create a field in each customer record for a code representing the customer value segment to which the customer belongs. Why? Once again, this creates the ability to automate marketing campaigns or personalization of a web site based on current and potential visitor value. Reps in a call center could also use the code, giving them a heads-up on the value of the customer to the company. As part of a customer retention plan, this code could determine how a rep responds to a customer request or problem. The bottom line on visitor and customer retention is this: identifying the current value of visitor and customer segments is moving from “best practice” to “no-brainer” status. The next leg up over your competition will be to use potential value metrics to make more profitable customer investment decisions for your company —Jim Novo and Eric T. Peterson Chapter 6, Web Measurement and the Online Retail Model 335 |

HACK #91 Distribute Reports Wisely Distribute Reports Wisely HACK Hack #91 #91 Don’t waste people’s time by sending out pages and pages of data Given all you’ve learned at this point in the book, I’m sure you’re thinking “That’s a lot of information to communicate.” You’re right, it is. Fortu- nately there are some simple, effective strategies for distributing reports that take advantage of what we know about people’s relationships with information. While I’m forced to use some generalities, experience tells me that often these assumptions hold up under scrutiny and can help you make better decisions about who gets what report when. Give the People What They Want, or Better Yet, What They Need People have a tendency, when volumes of data are available, to present volumes of data and let the reader sort it out; this is often the case with web measurement data. The problem with this strategy is that it assumes the reader will take the time to figure out which information is relevant to her; this is rarely the case in web measurement, usually because the data is foreign to most people. The best strategy to get people to invest their time is to give them the data they need to do their job and little else. If you take the time to figure out which data is most relevant to a person or group within your company and present that data in language they use and understand, you’ll see your efforts pay off, and inevitably your recipients will ask different and, hopefully, better questions. Use the Same Language Your Audience Uses Whenever Possible Rather than using the technical jargon used throughout this book, seriously consider translating your reports into the same language your business uses. Other than the fundamental definitions of page view, visit, unique visitors, and referrers [Hack #1], which are important to define clearly for your audience, make sure they understand each term—presenting web site activities in the lingua franca of your company is highly recommended. This is essentially an “if it ain’t broke, don’t fix it” recommendation. Also, keep in mind that some people are more visual than others and that images can augment language in ways you don’t expect. A Picture Says a Thousand Words While much of the data you’re collecting and analyzing is presented back to you in rows and columns, try and keep in mind the value of using images when presenting complex information. You don’t have to go out of your way to read Edward Tufte’s The Visual Display of Quantitative Information 352 Chapter 7, Reporting Strategies and Key Performance Indicators |

HACK Distribute Reports Wisely #91 (Graphics Press, 1992), but make use of visual elements whenever appropriate. For example, Figure 7-1 shows how raw data can be transformed into a much more dramatic presentation. Figure 7-1. Rich presentation in Visual Science’s Visual Workstation Of course, you don’t always have to go to such extremes. If you’re absolutely unable to visually represent the data, do simple things such as using up and down arrows to represent trends; color “good” numbers in green and “bad” numbers in red [Hack #92]; use bold, italics, and underline strategi- cally to highlight information; and leverage ratios to convey as much information as possible. Ratios Are Better than Counts While the difference between a ratio and a key performance indicator [Hack #94] is subtle at best, the difference between a count and a ratio is not. Would you rather know that 1,000 people bought something at your web site yesterday or that 16 percent of all visitors made a purchase? OK, sure, you probably want to know both, but the 1,000 people are presented out of context. “A thousand customers” is great news if you had only 10,000 visitors (a 10 percent conversion rate!) but slightly less good news if you had 1,000,000 visitors. Plus, there is tremendous value in comparing timeframes (this day versus this Chapter 7, Reporting Strategies and Key Performance Indicators 353 |

HACK #91 Distribute Reports Wisely day last week, for example), which is very difficult to do when presenting only a raw measurement. When in doubt, present both a meaningful ratio and the counts that support it, as shown in Figure 7-2. Figure 7-2. Key performance indicators and supporting metrics Distribute Reports Regularly One of the worst mistakes businesses make regarding web measurement data is to look at it infrequently or only on an ad hoc basis. Because this data is not the kind many have experience with, it’s important to keep people’s attention focused; the best way to do this is by getting reports in front of them frequently enough to generate familiarity and help them do their jobs better. If your marketing staffs tweak their advertising purchases on a weekly basis, make sure their marketing report is in their inbox at the beginning of every week. If your merchandising staffs rotate products every few days, make sure their merchandising reports are deliv- ered every day. You have to be especially careful regarding the timing of report delivery and consider the previous four recommendations. If you provide too much data, use confusing language, or make them scan ugly tables to mine for actionable information, it won’t matter how often you send the report, because it will always end up in the garbage. Conversely, if you do a good job of presenting the information people need to do their jobs in an easily understood format, you’ll be generating a report that people expect and rely upon for their ongoing success. 354 Chapter 7, Reporting Strategies and Key Performance Indicators |

HACK Use Key Performance Indicators #94 Use Key Performance Indicators HACK Hack #94 #94 Key performance indicators are a powerful way to present complex information that works to maximize the use of web measurement data within your organization. A key performance indicator (KPI) is any ratio that summarizes two or more important measurements and is tied directly to your business objectives [Hack #38]. Examples include ratios like your order conversion rate (orders divided by visits) or the average number of page views per visit: numbers that, when they change significantly, prompt someone to pick up the phone, send an email, instant message, or walk down the hall and say, “Something is going on; we need to look into this more deeply right away!” The use of key performance indicators is a powerful and advanced strategy that can dramatically increase your ability to get executive buy-in for your metrics reporting strategy [Hack #91]. A handful of really, truly useful key performance indicators is listed in Table 7-1. These are the kinds of useful ratios that are presented on a daily basis to captains of industry like Michael Dell, Jeffery Bezos, and Meg Whit- man: CEOs who clearly get the power of the Internet and understand that every minute counts in an increasingly competitive world. Table 7-1. Really, truly useful key performance indicators Order conversion rate Buyer conversion rate Cart conversion rate Checkout start rate Revenue per visit Revenue per visitor Average order value Visits per visitor Page views per visit Percent committed visitors Lead conversion rate Home page bailout rate Average number of items Average time spent on site Percent file take per purchase While Table 7-1 provides a handful of examples, there are hundreds of other potentially valuable measures. Your central challenge is to figure out which ones are best for your business. Here are some recommendations to consider: Refer back to your business objectives Any ratio that speaks directly to your company’s business objectives [Hack #38] and will drive action is a good KPI, as are any ratios that are direct measurements of key activities associated with your business goals. Figure out which indicators are really “key” George Orwell once wrote that “all numbers are created equal, but some are more equal than others” (or something like that); clearly Mr. Orwell was a web measurement guru in his spare time. While the KPIs most valuable to specific business models are covered later in this chapter, determining which numbers are “more equal than others” is a great Chapter 7, Reporting Strategies and Key Performance Indicators 361 |

HACK #94 Use Key Performance Indicators place to start. In general, any number or ratio that senior managers ask about on a regular basis should be considered important. Make sure your indicators promote action The best KPIs are those that, when people look at them and realize they’ve gone down from week to week, make people freak out and call meetings. The numbers that make people the most nervous are the best candidates, always. Conversely, if you’re thinking about a number but cannot think of any action you would take if that number absolutely tanks, set that number aside. When in doubt, simply consult the hacks describing specific key performance indicators for online retail [Hack #96], advertising and content [Hack #97], customer support [Hack #98], and lead generation [Hack #99]. Best Practices for Deﬁning Key Performance Indicators Assuming you’re still nodding your head and you’re thinking to yourself, “yeah, I need to make up some KPIs right away,” here are a handful of best practices that you should follow. Use KPIs to drive action. The most important thing any key performance indicator does is get someone to take a closer look at your visitor’s behavior. Since you’ll be using KPIs to compare data day to day or week to week, any time you see a strong decrease or a surprising increase, you need to be ask- ing yourself, “why did that happen?” and, “what impact will that change have on my business?” Make sure that you do everything in your power to highlight significant changes, using colors, fonts, and in-your-face warning messages when necessary (Figure 7-5). Present KPIs visually whenever possible. You should give serious consideration to how you present the information, and make an attempt to, well, make it interesting. Strange as it sounds, overburdened senior executives often respond to visual representations that present complex information in a simple, effective format. Some analytics vendors allow you to present KPIs using tachome- ters, thermometers, trended graphs, and the like, as illustrated in Figure 7-6. Use the language of the business to increase familiarity. Another nice benefit of using key performance indicators is that you’re able to use your own words to describe the numbers, not the words used in your measurement application. It sounds simple, but this can be very important; you don’t want to force people inside your organization to learn new names for ideas they’re already familiar with. For example, if people are familiar with “average sale value” (ASV) not 362 Chapter 7, Reporting Strategies and Key Performance Indicators |

HACK Use Key Performance Indicators #94 Figure 7-5. Key performance indicator worksheet “average sale price” (ASP) use average sale value in your report. Familiarity with the data lowers the barrier to understanding and use. Explain the how and why of KPIs. Since the use of KPIs is pretty advanced, many of the folks you provide them to will be unfamiliar and will require further explanation. Two simple things you can do to help are to provide personalized training and a glossary with every KPI report. The training will allow folks to ask questions (and help you determine whether they get it), and the glossary will save you time because these folks will have something to refer to (other than you) if they forget what you told them. While it will take a little extra work to build KPIs and get them implemented into your reporting program, experience shows it’s well worth it. Everything you do to make web measurement data more palatable helps. By maximizing the information content and presentation of the numbers you provide, you can dramatically increase people’s interest and use of the data. Chapter 7, Reporting Strategies and Key Performance Indicators 363 |

HACK #94 Use Key Performance Indicators Figure 7-6. KPI dashboard 364 Chapter 7, Reporting Strategies and Key Performance Indicators |

Index & (ampersand), in query string, 91 analysis of data = (equal sing), in query string, 91 benchmarking against external ? (question mark), preceding query data, 358 string, 91 benchmarking against yourself, 358 benchmarking, reasons not to, 357 communicating throughout A company, 349–350 abandonment, 237 comparisons to competitors, 356 during checkout process, 315, 322 determining if results are positive or rate of, 237 negative, 355–357 of shopping cart, 317 Excel for, 351 ABC Interactive (ABCi), 84 key performance indicators access logs, 70 for, 360–364 acquisition, visitor, 152 tools for, 356 activity reports, 262 (see also key performance indicators; advertising (see marketing) reports) advertising sites, key performance anti-adware applications, cookies indicators for, 370–376 and, 59 affiliate marketing, 184–188 anti-spyware applications agent logs, 70 cookies and, 59 agents, excluding data from, 83–86 disallowing third-party cookies, 58 Akamai CDN platform, 90 AOV (average order value), 366 Akamai geo-targeting service, 301 application examples (see RSS tracking ampersand (&), in query string, 91 application; web measurement Analog, 34 application) configuring, 39 application servers, logfiles produced downloading, 38 by, 82 installing, 38 application service provider model LOGFORMAT command, 134 (hosted services model), 14 processing logfiles using, 38–40 application usage reports, 262 running, 39 ASP model (hosted services model), 14 We’d like to hear your suggestions for improving our indexes. Send email to index@oreilly.com. 391

attrition rate, 237 client-side cache, 87 authenticated username, 80 data not collected because of, 77 identifying repeat visitors using, 64 JavaScript page tags and, 71, 90 identifying unique visitors using, 63 page caching, causing false authentication server, 79 entries, 232 average order value (AOV), 366 server-side cache, 87 AWStats, 34 cache-control.inc file, 88 caching device, 87 call to action links, 225 B campaign ROI, determining, 123 B2B (business-to-business) visitors, 262 cart (see shopping cart) B2C (business-to-consumer) cart-add to purchase conversion visitors, 262 rate, 368 banner advertising, 163–166 CDN (content delivery network) BBBOnline privacy certification, 98, 99 data lost when using, 77, 78 benchmarking improving performance using, 90 against external data, 358 checkout process, 247 against yourself, 358 measurements for, 320–323 reasons not to, 357 streamlining, 315 services for, 358 usability of, 240–243 best practices for web site checkout to purchase conversion measurement, 10–13 rate, 368 \"black holing\", DNS, 77 CLF (Common Log Format), 81 Blogdigger, 192 Clicklab, 37 Bloglines, 192 clicks blogs (see weblogs) cost-per-click (CPC), 152, 175, 179 bookmarking, tracking, 282–284 pay-per-click search engines, 181, broad matching of keywords, 181 205–209 broadband connection, tracking, 222–223 determining, 271–276 value of, measuring, 219–221 browser overlays, 15, 220, 245–248 clickstream, 6 browser (see web browser) click-through rate (CTR), 152 Bugnosis, 60, 112 for affiliate marketing, 185 business objectives for banner advertising, 165 identifying, 156–158 for email, 168 key performance indicators tied for paid search engine to, 360 marketing, 174 business sites, key performance click-to-conversion rate (CTC), of indicators for, 381–386 banner advertising, 166 business-to-business (B2B) visitors, 262 ClickTracks business-to-consumer (B2C) features of, 14, 15, 36 visitors, 262 support provided by, 53 buyer conversion rate, 154, 161, 368 client-side cache, 87 client-side page tags (see JavaScript page C tags) code examples cache in this book, using, xx browser cache, 87 web site for, 4 \"busting\" (forcing to request pages (see also RSS tracking application; from server), 87–90 web measurement application) CDN, data lost when using, 77, 78 392 | Index

color depth, 289 for paid search engine Combined Log Format, 70, 82 marketing, 175 commerce data, not collecting, 72 for registrations, 375 commerce (see retail, online) scenario conversion, 161 committed visits, 374, 385 search to purchase conversion common buyers, 336 rate, 369 Common Log Format (CLF), 81 for searches, 254 compact policy (CP), 58, 102, 103 for subscription sign-up, 375 competitive analysis tools, 356 value events, 200–203 competitors, comparing results to, 356 cookie dropping, 108 ComScore, 356 cookies configurators, measuring use of, 120 accuracy of, 58, 59 connection type alternatives to, 62–69 determining, 271–276 custom variables stored in, 118–122 significance of, 288 data to use, 288 contact information, xxi for differentiating new and returning content consumption reports, 262 customers, 341–342 content delivery network (CDN) improving data accuracy data lost when using, 77, 78 using, 56–59 improving performance using, 90 privacy and, 58 content groups, naming, 73–75 returned by JavaScript page tags, 108 content length, 80, 170 types of, 56 content of web pages, wording used when not to use, 62 in, 223–227 Coremetrics, 15, 16, 53 content sites, key performance Coremetrics LIVEmark, 359 indicators for, 370–376 cost data, integrating with web conventions used in this book, xix measurement data, 123 conversion events and rates, 153, cost-per-acquisition (CPA), 153, 166 158–162 cost-per-click (CPC) for banner ads, 165 for organic search results, 179 buyer conversion rate, 154, 161, 368 of marketing campaign, 152 cart-add to purchase conversion of paid search engine marketing, 175 rate, 368 cost-per-converion (CPC), 153 checkout to purchase conversion counts, using in reports, 353 rate, 368 CP (compact policy), 58, 102, 103 click-to-conversion (CTC) rate, 166 CPA (cost-per-acquisition), 153, 166 cost-per-conversion (CPC), 153 CPC (cost-per-click) for email marketing, 169 for organic search results, 179 for \"information find\" support of marketing campaign, 152 visits, 380 of paid search engine marketing, 175 as key performance indicators, 160 CPC (cost-per-conversion), 153 lead generation conversion rate, 382 crawlers, excluding data from, 83–86 measuring through multiple cross-sell data, 334–337 goals, 199–203 CTC (click-to-conversion rate), of for multi-step processes, 237 banner advertising, 166 multi-step conversion funnels, 162 CTR (click-through rate), 152 new and returning visitor conversion for affiliate marketing, 185 rates, 369 for banner advertising, 165 order conversion rate, 154, 161, 366, for email, 168 368 for paid search engine for organic search results, 179 marketing, 174 overall conversion, 160 custom links, 219 Index | 393

custom variables, 118–122 date and time (timestamp), 80 direct marketing and, 120 Deep Log Analyzer, 36 query string data collected in, 94 delivery of web site, 27 uses of, 119 delivery type of vendor, 13, 14 Customer Paradigm’s P3P Privacy Policy demographic data, 267, 268 Creation, 101 determining with custom customer support reports, 263 variables, 119 customer support sites, key performance tracking, 294–300 indicators for, 376–381 visitor segmentation based on, 297, customers 299 latency of, 327, 330 designated marketing area (DMA), 269, new, 340–343 301 recency of, 327–329 destination pages, 234 registration data, integrating, 123 Digital Envoy, 301 retention rate for, 367 direct marketing, custom variables returning, 340–343 for, 120 satisfaction data, integrating, 123 DMA (designated marketing area), 269, (see also visitors) 301 DNS \"black-holing\", 77 D using to make third-party cookies data look like first-party accuracy of, improving with cookies, 58 cookies, 56–59 document headers, requesting that compared to key performance document not be cached, 88 indicators, 364 DOM (Document Object integration of, best practices Model), 114–117 for, 122–126 beacon placement for, 114 sources of, 22–25 browser width, determining, 116 storage of, 76, 78 form analysis using, 117 (see also key performance indicators; form entry errors, tracking, 116 reports) information provided by, 114 data collection download managers, tracking collecting all data, reasons not to, 72 downloads from, 306 form data, 92 downloads how long to keep data, 73 interrupted, tracking, 130 lost data, preventing, 78 not collecting data about, 72 lost data, reasons for, 75–78 time of, affecting measurements, 228 PII (personally identifiable tracking, 304–307 information), 97 dynamic URLs, errors with, 133 robots, excluding data from, 83–86 for RSS tracking application, 45–50 E spiders, excluding data from, 83–86 types of data to eliminate, 72 email marketing, 167–172 users accessing collected data, 96 date and time of delivery, 171 users opting in or out of, 96 format and layout of email, 170 vendor mechanism for, 13, 14 integrating email campaign data, 124 for web measurement length and tone of email, 170 application, 40–45 return address of, 171 what data to collect, subject line of, 171 determining, 52, 69–73 394 | Index

encrypted web traffic, network data fonts used in this book, xix collector used with, 31 form analysis, with DOM, 117 entry pages, 232, 236 form data, collecting, 92 entry pages report, 232, 236 form entry errors, tracking, 116 equal sign (=), in query string, 91 frequency of visits by a visitor, 324–327 error checking, for web measurement FunnelWeb, 35 application, 388 error logs, 70 G errors, tracking, 119, 129–133 geographic distribution of ETL (extract, transform, and load) tool, visitors, 301–304 Sane Solutions, 16 geographic distribution report, 301 exact matching of keywords, 181 geographic segmentation, 338–340 example applications (see RSS tracking geo-targeting application; web measurement in web measurement application) application, 308–310 examples (see code examples) services for, 301 Excel (Microsoft) getsites 1.4 log analyzer, xv distributing data using, 351 gift list, shopping cart used as, 319 populating with web measurement Google AdSense, 184 data, 15 gross margin contribution, executive buy-in, 11 calculating, 123 exit pages, 233 ratio of entry pages to, 236 H search return page as, 254 exit pages report, 233, 236 hacking, xvi external/business-to-consumer (B2C) hacks, contributing, xxi visitors, 262 hard bounces of email, 167 extract, transform, and load (ETL) tool, HitBox Professional, 36 Sane Solutions, 16 hits, 5 extranet, measuring traffic to, 126–129 Hitwise, 356 Holland’s OneStat, 282 F home page as exit page, 234 fallout reports, 238 not as entry page, 232 false entries, 232 (see also landing pages) false exits, 233 hosted services model, 14 favicon.ico file, 283 hostname, remote, 79 favorites, tracking, 282–284 HTTP requests, tracking downloads Federal Trade Commission report on using, 305 Privacy Online, 100 HTTP status codes, 80, 131 FeedBurner, 149 HTTP: The Definitive Guide, 70 Feedster, 192 Hughes, Kevin (getsites 1.4), xv file downloads (see downloads) human resources (see team for web site FILESMATCH directive, Apache, 89 measurement) Fireclick, 15, 16, 53 hyperlinks (see links) Fireclick Index, 358 Firefox, controlling cookies using, 59 I first-party cookies, 57 advantages of, 59 IAB (Interactive Advertising Bureau), 99 for differentiating new and returning \"page views\" defined by, 6 customers, 341–342 \"unique visitors\" defined by, 7 when to use, 58, 59–62 \"visits\" defined by, 6 Index | 395

IAjapan’s Privacy Policy Wizard, 101 J IAPP (International Association of Java, whether to use, 289 Privacy Professionals), 100 JavaScript IBM P3P Policy Generator, 101 errors, tracking, 132 IBM SurfAid, 15, 16, 53 version of, whether to track, 290 icons whether to use, 289 for favorites folder, 283 JavaScript page tags, 23, 106 used in this book, xx caching and, 71, 90 IIS Logfile Format, 82 click counting problems, image requests resolving, 208 excluding form logfile, 70 custom variables in, 118–122 for DOM, 114 errors tracked in, 131 in JavaScript page tags, 108, example of, 108–111 111–114 execution process of, 107 images image requests used by, 108, icon for favorites folder, 283 111–114 in reports, 352, 362 lost data, reasons for, 76 setting to never expire, 89 missing, causing false entries, 232 implementation naming pages programmatically, 73 optimizing, 52–56 passing demographic data to, 296 problems with, resolving, 52, 55 performance of, 71 reports to implement first, 54 query string data, collecting, 94 timeline for, 52 responsibility for, 78 vendors, working with during, 53 RSS tracking using, 193 impressions, of banner ads, 163 session cookies and, 56 improvement process, 12 variables in, 108 IndexTools, 36 for web measurement Information Architecture and the World application, 40–45 Wide Web, 75 what data to collect using, 71 \"information find\" customer support when to use, 24 visits, 380 Interactive Advertising Bureau (see IAB) K Interactive Audience Measurement and Advertising Campaign key event participation, Reporting and Audit determining, 119 Guidelines, 6, 84 key performance indicator report, 160, internal campaigns, 243–245 162 internal searches, 253–255, 256–260 key performance indicators (KPIs), 70, internal/business-to-business (B2B) 360–364 visitors, 262 for advertising sites, 370–376 International Association of Privacy benchmarking against yourself Professionals (IAPP), 100 using, 358 intranet, measuring traffic to, 126–129 compared to measurement data, 364 IP address for content sites, 370–376 determining unique visitors conversion events as, 160 using, 62 for customer support sites, 376–381 of requestor, 79 determining, 362 IP2Location, 301 for lead generation, 381–386 item abandonment, 317 for online retail, 365–370 terminology used to describe, 362 396 | Index

useful, list of, 361 logfiles, application server, 82 visitor intent and, 27 logfiles, web server (see web server visual presentation of, 362 logfiles) keywords, paid, compared to organic LOGFORMAT command, Analog, 134 search queries, 180–183 look-to-book ratio, 314, 317 known visitors, 22, 260–264 loss, for organic search results, 180 identifying, 260 reports for, 262 M types of, 261 Macromedia Flash Local Shared KPIs (see key performance indicators) Objects, 65–69 macro-ROI, 151 L marketing landing pages advertising sites, key performance stickiness of, 168 indicators for, 370–376 unique, 188–191, 206 affiliate marketing, 184–188 language (wording) banner advertising, 163–166 used in email, 170 business objectives and, 156–158 used in reports, 352 click counting problems, used in web pages, 223–227 resolving, 205–209 (see also terminology of web site conversion events and measurement) rates, 158–162 languages conversion, multiple goals used to segmenting visitors by, 292–294 measure, 199–203 translation, when to use, 291 cost data, integrating with web used by visitors, 290–292 measurement data, 123 latency of customers, 327, 330 email marketing, 167–172 lead generation conversion rate, 382 expenditures, optimizing, 313 lead generation, key performance internal campaigns, 243–245 indicators for, 381–386 mini-sites, 190 lifetime value of a campaign, 156 offline, geographic segmentation lifetime value of visitors, 324–327, and, 338–340 331–334 organic search results, 177–180 from organic search results, 180 paid keywords compared to organic from paid search marketing, 175 queries, 180–183 linear scenarios, 161 paid search engine links marketing, 173–177 broken, monitoring, 130 pay-per-click model, 205–209 as calls to action, 225 referring URLs, 203–205 click-through rates superimposed response lift, 339 on, 246 retention and, 155 custom links, 219 success lift, 339 determining if clicked, 119 terminology for, 150, 151–156 placement of, 247 unique landing pages, 188–191, 206 as points of resolution, 225 visitor loyalty segments, 209–212 text for, 227 visitor reach and acquisition, 152 tracking clicks on, 219, 222–223 visitor segmentation, 195–199 list, shopping cart used as, 319 web measurement application Local Shared Objects, 65–69 example using, 213–217 Log Parser, 35 MaxMind, 301 Log Parser Toolkit, 35 measurements (see data; reports; web logfile analysis tools, xv, 38–40, 82 site measurement) Index | 397

median, 229 organic search results META tags, requesting that document contrasting queries with paid search not be cached, 88 engine keywords, 180–183 metrics (see data; reports) measuring, 177–180 micro-ROI, 151, 154 outsourced model (hosted services Microsoft Excel (see Excel) model), 14 Microsoft Log Parser, 35 overall conversion, 160 Microsoft Log Parser Toolkit, 35 mini-sites, 190 P mistakes, tracking, 129–133 P3P compact policy, 58, 102, 103 mobile devices, providing content P3P (Platform for Personal for, 289 Preferences), 98, 100 modem connection, P3P policy generators, list of, 101 determining, 271–276 P3P privacy policy mod_log_config file, Apache, 70 creating, 100–106 monitoring port, 30 page tag usage explained in, 113 mostly anonymous visitors, 21 resources for, 106 multi-step conversion funnels, 162 returned by JavaScript page tags, 108 multi-step processes, 237–240, 247 P3PEdit, 101 (see also checkout process) p3p.xml file, 102 packet sniffers (see network data N collectors) NAI (Network Advertising page allocation, 220 Initiative), 99 page allocation report, 221 NAI Web Beacon Guidelines, 100 page hits, 5 named user analysis, 128 page participation, 220 NCSA Common format, 81 page participation report, 221 NCSA Extended log format, 70 page tags (see JavaScript page tags) NetTracker Professional, 37 page views, 6 NetTracker (Sane Solutions), 16 average pages viewed per visit, 371, Network Advertising Initiative 376 (NAI), 99 defining in reports, 352 network data collectors (packet number per visit, analyzing, 355 sniffers), 23 pages (see web pages) capabilities of, 28 paid search engine marketing, 173–177 in switched environment, 29 contrasting keywords with organic when to use, 24, 27–31 search queries, 180–183 nonlinear scenarios, 162 differentiating destination URL from organic search, 178 pay-per-click search engines, 181, O 205–209 objectives for web site measurement, performance identifying, 10 cache busting and, 89 offline marketing, 338–340 content delivery network (CDN) Omniture, 14, 15, 53 and, 90 Omniture’s SiteCatalyst, 282 of JavaScript page tags, 71 OneStat (Holland), 36, 282 of web pages, 207 online retail (see retail, online) of web servers, 130 operating system, whether to track, 290 of web site, 269–271 order conversion rate, 154, 161, 366, web traffic and, 133 368 (see also key performance indicators) 398 | Index

persistent cookies, 56, 56–59 up-selling, 335 persistent shopping carts, 319 publications PII (personally identifiable Federal Trade Commission report on information), 97 Privacy Online, 100 Platform for Personal Preferences (see HTTP: The Definitive Guide, 70 P3P) Information Architecture and the plug-ins World Wide Web, 75 data about, not collecting, 72 Interactive Audience Measurement determining whether and Advertising Campaign installed, 285–287 Reporting and Audit ignoring data about, 289 Guidelines, 84 point-of-resolution links, 225 \"Privacy Online: Fair Information policy reference file, 102 Practices in the Electronic policy validators, 103 Marketplace\", 95 power users, training, 55 The Visual Display of Quantitative privacy Information (Tufte), 352 anti-spyware applications, 58, 59 Web Caching, 87 certification through third party, 98 Web Performance Tuning guidelines for, 95 (Killelea), 208, 269 importance of considering, 95 purchase process, usability of, 240–243 Local Shared Objects and, 69 pushpin icon, xx P3P technology for, 98 resources for, 98 Q security and, 96 query strings violations of, finding, 60 custom variables in, 118–122 \"Privacy Online: Fair Information using in web site Practices in the Electronic measurement, 91–94 Marketplace\", 95 question mark (?), preceding query privacy policy string, 91 collecting data about visitor access Quova, 301 of, 100 compact policy (CP), 58, 102, 103 R cookies explained in, 58 cookies prohibited by, 61 ratios, using in reports, 353 creating using P3P reach, visitor, 152 technology, 100–106 recency of customers, 327–329 JavaScript page tag usage explained red eye icon, 105, 113 in, 113 referrer logs, 70 notice of, 96 referrer report, 228 omitting, consequences of, 105 referring domains report, 203 writing, 97 referring URLs (referrers), 8, 203–205 PrivacyBot.com, 101 defining in reports, 352 process reports, 238, 240 in web server logfile, 80 product selection engines, measuring use registration data, integrating, 123 of, 120 registration status, determining, 119 products remote host, 79 average order value (AOV), 366 rendering time, affecting time cross-selling, 334–337 measurements, 228 product placement, 313–315 repeat visitors, determining, 64 Index | 399

Report Magic, 40 checkout process reports measurements, 320–323 communicating throughout commerce data, not collecting, 72 company, 349–350 cross-sell data, 334–337 for customer support, 263 customer latency, 327, 330 distributing, 352–354 customer recency, 327–329 entry pages report, 232, 236 frequency of visitors, 324–327 Excel used to analyze, 351 key performance indicators exit pages report, 233, 236 for, 365–370 fallout reports, 238 lifetime value of visitors, 324–327, geographic distribution report, 301 331–334 images in, 352 look-to-book ratio, 314 key performance indicator marketing expenditures, report, 160, 162 optimizing, 313 for known visitors, 262 measurements, guidelines for most important, implementing using, 312–316 first, 54 new customers, 340–343 page allocation report, 221 offline marketing for, 338–340 page participation report, 221 product placement, 313–315 process reports, 238, 240 returning customers, 340–343 ratios in, 353 shopping cart referrer report, 228 measurements, 316–320 referring domains report, 203 for web measurement for RSS tracking application, 344–348 application, 138–149 retention sales analysis reports, 263 of customers, 367 segmentation report, 197 of visitors, from marketing single-access pages report, 234, 236 campaign, 155 terminology used in, 352 of visitors, in multi-step process, 237 viewing on pages (browser (see also visitors, loyalty of) overlays), 245–248 return on investment (see ROI) visitor language report, 290 revenue for web measurement as key performance indicator, 364 application, 386–389 for organic search results, 180 request, 80 per visit or visitor, 153 Request for Proposal (RFP) for vendor RFC931, 79 selection, 32–33 RFP (Request for Proposal) for vendor requirements for using hacks, 4 selection, 32–33 resolution, 276–279, 287, 289 robots resources (see publications; web sites analyzing data from, 86 (resources)) excuding data from, 83–86 response, banner ads, 163 list of known robots, 84 response lift, 339 robots.txt file, 85 response rate, 153 ROI, 150 of email, 168 determining, 123, 126 of paid search engine marketing, 174 micro-ROI, 151, 154 response time, 269 rolling ROI, 155 results of web site measurement (see RSS data; reports) measuring readership of, 191–195 retail, online, 311 tracking activity for, 3 400 | Index

RSS tracking application shopping cart data collection for, 45–50 abandonment of, 317 parsing logfile and generating cart overlap, 336 reports, 138–149 cart-add to purchase conversion rate, 368 measurements for, 316–320 S not used as shopping cart, 319 sales persistent, 319 as key performance indicator, 364 shopping (see retail, online) per visitor, 366 single-access pages, 234 sales analysis reports, 263 single-access pages report, 234, 236 Sane Solutions, 15, 16, 53 site hits, 5 satisfaction data, integrating, 123 SiteCatalyst (Omniture), 15, 282 scenario conversion, 161 software model, 13 screen resolution, 276–279, 287, 289 Speedera CDN platform, 90 screw icon, xx spiders script errors, 130 analyzing data from, 86 Search Engine Marketing Professional excluding data from, 83–86 Organization (SEMPO), 99 list of known spiders, 84 search engines unfiltered spider activity, 166 content of web page and, 226 split-path testing, 119, 248–253 hits from, 5 SSL, network data collector used local search features by, 304 with, 31 paid search engine staff (see team for web site marketing, 173–177 measurement) pay-per-click, 181, 205–209 StatMarket (WebSideStory), 282 search to purchase conversion rate, 369 stickiness of web pages searches first page, 374, 380, 385 internal searches, 253–260 landing pages, 168 in intranets or extranets, success lift, 339 measuring, 127 SurfAid (see IBM SurfAid) organic compared to paid switched environment, network data keywords, 180–183 collectors and, 29 organic search results, 177–180 Syndicate IQ, 149 search terms, custom variables collecting, 119 T zero yield searches, 255, 256, 379 zero-results searches, 255–260, 378 tags, data collection using (see security, 96 JavaScript page tags) (see also privacy) targeted email campaign data, segmentation report, 197 integrating, 124 segmentation (see geographic team for web site measurement segmentation; visitor building, 11 segmentation) list of members, 17–20 SEMPO (Search Engine Marketing technographic data, 267 Professional Organization), 99 in web measurement server-side cache, 87 application, 308–310 session cookies, 56 not collecting, 72 session identifier, identifying unique uses of, 268 visitors using, 64 which data to ignore, 289 sessionization, 62, 134 which data to use, 287, 289 Shared Objects, 65–69 Technorati, 192 Index | 401

terminology of marketing, 150, usability 151–156 for checkout process, 240–243 terminology of web site entry pages and, 232 measurement, 4–10 exit pages and, 233 for key performance indicators, 362 for internal searches, 253–255 using in reports, 352 internal campaigns and, 243–245 The Visual Display of Quantitative for multi-step processes, 237–240 Information (Tufte), 352 single-access pages and, 234 thermometer icons, xx split-path tests for, 248–253 third-party cookies, 57 testing, 218, 219 appearing as first-party cookies, 58 textual content’s effect on, 223–227 determining whether they’re being time spent on site and, 227–231 used, 60 for web measurement when to use, 60 application, 264–266 third-party privacy certification, 98 user agent, 81 time spent on web site, user agent strings, determining unique measuring, 227–231 visitors using, 63 timeline for implementation, 52 username, authenticated, 80 timestamp, 80 identifying repeat visitors using, 64 totally anonymous visitors, 21 identifying unique visitors using, 63 tracked click-through of email, 168 users tracked opens of email, 167 access of collected data by, 96 tracking URLs, 178 named user analysis, 128 training of users, 55 opting in or out of data transactional pages, 247 collection, 96 transition pages, 234 power users, training, 55 translation, when to use, 291 response to web site, 27 TRUSTe privacy certification, 98, 99 unique user identifier, 120 Tufte, Edward (The Visual Display of Quantitative V Information), 352 value events, 200–203 Turner, Stephen, Dr. (Analog variables developer), 38 custom, 94, 118–122 in JavaScript page tags, 108 U vendors unfiltered spider activity, 166 data collection mechanism of, 13, 14 unique landing pages, 188–191, 206 data lost by, 75 unique opens of email, 167 delivery type of, 13, 14 unique user identifier, 120 free, finding, 34 unique visitors, 7 history of emergence of, 2 defining in reports, 352 hosted services model used by, 14 determining, 62 inexpensive, finding, 35 unsubscribes, from email inexpensive, when to use, 33 marketing, 168 limiting time data is kept, 73 up-selling, 335 problem solving with, 52, 55 Urchin, 15, 16, 37, 53 representation of, in this book, xix URLs RFP for, 32–33 dynamic, errors with, 133 selecting, 13–17 referring URLs, 8, 80, 203–205 software model used by, 13 tracking URLs, 178 support provided by, 53, 55 402 | Index

working with during visits, 6 implementation, 53 average visits per visitor, 372, 382 version of browser, committed visits, percent of, 374, determining, 279–282 385 visitor language report, 290 defining in reports, 352 visitor segment value matrix, 331–334 expiration of, 228, 233 visitor segmentation, 27, 195–199 interested visits, percent of, 384 based on demographics, 297, 299 last page in, measuring time for, 228 by language, 292–294 of length less than 90 seconds, 369, lifetime value and, 331–334 374, 380, 384 visitor loyalty and, 210–212 length of, analyzing, 355 for web measurement number of, analyzing, 355 application, 387 searches per visit, 254 visitors tracking, example of, 134–138 assigning to test group for split-path visual presentation testing, 251 of key performance indicators, 362 average visits per visitor, 372, 382 of report data, 352 frequency of visits by, 324–327 Visual Sciences, 15, 16, 53 geographic distribution of, 301–304 intent of, determining, 25–27 W known visitors, 260–264 W3C document on logging control, 81 languages used by, 290–292 W3C Extended Logfile Format, 82 lifetime value of, 324–327, 331–334 W3C P3P Platform for Privacy lifetime value of, from organic search Preferences, 100 results, 180 W3C policy validator, 103 lifetime value of, from paid search Web Analytics Association, 19 marketing, 175 web analytics, compared to web site losing in multi-step process measurement, xvii, 3 (abandonment), 237 Web Analytics Forum, 19 loyalty of, 209–212, 332 web beacons, 111–114 loyalty of, bookmarking web browser indicating, 282–284 browser cache, 87 motivations of, determining, 224 browser overlays, 15, 220, 245–248 new and returning visitor conversion errors with, 133 rates, 369 height of, 289 new and returning visitors, percent plug-ins, determining whether of, 366, 372, 377, 382 installed, 285–287 new to returning visitors, ratio plug-ins, ignoring data about, 289 of, 367, 373, 383 plug-ins, not collecting data reach and acquisition, 152 about, 72 registration status of, type of, 287, 289 determining, 119 version of, 279–282 repeat visitors, determining, 64 width of, 116, 287 retention rate, 155, 237 web bugs, 111–114 sales per visitor, 366 Web Caching, 87 tracking, example of, 134–138 web data analyst types of, 20–22 finding, 19 unique visitors, 7, 62 skills required for, 18 (see also customers) VisitorVille, 36 Index | 403

Web Measurement and Analysis logfile analysis tools, xv, 38–40 Forum, 17 lost data, reasons for, 77 web measurement application not using for data collection, 42 commerce data in, 344–348 passing demographic data to, 296 data collection for, 40–45 problems with, 22 determining visits and remote, 389 visitors, 134–138 RSS tracking using, 193 error checking for, 388 session cookies and, 56 marketing data, 213–217 types of, 70, 81 multi-session tracking for, 389 what data to collect using, 70 remote logfile for, 389 when to use, 24 reporting, 386–389 web site hits, 5 technographic data web site measurement tracking, 308–310 adoption and interest in, tracking exits from, 389 increasing, 247 usability data, 264–266 best practices for, 10–13 visitor segmentation for, 387 business objectives and, 156–158, web pages 360 aborted, tracking, 130 communicating results alternative, randomly displaying from, 349–350 (split-path testing), 248–253 compared to web analytics, xvii, 3 content of, 223–227 executive buy-in for, 11 entry pages, 232, 236 history of, xv, 2 exit pages, 233, 236, 254 hosted services model for, 14 landing pages, 168, 188–191, 206 implementing, 52–56 multi-step processes using, 237–240 improvement process for, 12 naming, 73–75 objectives for, identifying, 10 normal viewing time for, reasons for, 1 estimating, 230 requirements for using hacks, 4 page allocation, 220 robots perforrming, 84 page participation, 220 software model for, 13 performance of, 207, 228 team for, 11, 17–20 single-access pages, 234 terminology used by, 4–10, 352 stickiness of, 168, 374, 380, 385 what it is not, 2 time spent in, measuring, 227–231 web sites transactional pages, 247 advertising, key performance transition pages, 234 indicators for, 370–376 transmission and load time of, 228 bookmarking of, tracking, 282–284 value of, measuring, 219–221 business, key performance indicators Web Performance Tuning for, 381–386 (Killelea), 208, 269 content, key performance indicators web server for, 370–376 errors with, 133 customer support, key performance network data collector running indicators for, 376–381 on, 31 delivery of, user’s response to, 27 performance problems with, 130 lead generation, key performance web server logfiles indicators for, 381–386 contents of, 79–81 mini-sites, 190 errors recorded in, 131 performance of, 269–271 excluding image requests from, 70 privacy violations in, finding, 60 404 | Index

time spent in, average, 355, 373, Web Analytics Association, 19 377, 383 Web Analytics Forum, 19 time spent in, measuring, 227–231 Web Measurement and Analysis web sites (resources) Forum, 17 Analog, 38 weblog tracking tools, 149 benchmarking services, 358 WebSideStory’s StatMarket, 282 Blogdigger, 192 WebsiteOptimization.com, 208 Bloglines, 192 Wikipedia, 87 Bugnosis, 60 weblogs CDN platforms, 90 measuring readership of, 191–195 code examples, 4 tracking tools for, 149 competitive analysis tools, 356 WebSideStory, 14, 15, 53 DMAs (designated marketing WebSideStory’s StatMarket, 282 areas), 301 WebSiteOptimization.com, 208 Feedster, 192 Web-Stat, 36 for this book, xxi WebSTAT Premium, 36 free web measurement solutions, 34 Webstats, 34 \"geo-targeting\" services, 301 WebTrends, 2, 14, 15 Holland’s OneStat, 282 \"hits\" defined by, 5 IAB (Interactive Advertising support provided by, 53 Bureau), 84 WebTrends 7 Small Business inexpensive web measurement Edition, 37 solutions, 36 Wikipedia, 87 log file formats, 81 Williams, Roy H. (effect of words P3P policy generators, 101 throughout history), 223 P3P privacy policies, 106 wish list, shopping cart used as, 319 policy validators, 103 WWWStat, 34 privacy information, 98 Report Magic, 40 Z RFC931 FAQ, 79 zero yield searches, 255, 256, 379 robots and spiders, list of, 84, 85 zero-results searches, 255–260, 378 Technorati, 192 vendors, history of emergence of, 2 W3C document on logging control, 81 Index | 405

Web Site Measurement Hacks Sample

1 comments

Notes on slide 1

1 Favorite