What Gets Measured Gets Done – Tom Peters (Borrowed from Booz Allen Hamilton)
Allign your measurements to the business objectives
Think about brand as well as transactional measures
Just because something can be measured doesn’t mean that it should be measured
Beware the Law of Unintended Consequences “any purposeful action will produce some unintended consequences.” Credited from anybody from Robert K. Merton to Adam Smith
QUICK CASE STUDY
Yahoo! Answers launched in December 2005 in the US
Knowledge search: relies on audience participation
Based on successful Korean business model championed by Naver
Rewards points to question responders
Measurement fell down in two areas:
Focus on unique user numbers, rather than the economic value of the user
Struggling to get the points system right for participants
Non-virtuous circle ensued
High community management costs
Harder to monetise ad inventory as it attracts the wrong people
Move to reduce community costs by flipping management over to users (weighting based on points system) brought in further criticism
ON NETWORK MEASURES
Number of returning vistors
Time spent on site
Average cost of acquisition per user
Value of transaction
OFF NETWORK MEASURES
Google Adword account information
Cost per click
Relative performing words
Click through rate (advertising / SEM)
Track variations in SEM copy
Value of your brand as key words over time
Tone of voice
Inbound links from social media
OFF NETWORK MEASUREMENT: SILVER BULLET (OR NOT)
Google Analytics and many of the measuring tools provide dashboards
Very PowerPoint friendly
How does the dashboard map to your objectives?
If it doesn’t map to your objectives what is its value?
BACKBONE OF MEASUREMENT AND TARGETING – COOKIES
HTTP cookies , or more commonly referred to as Web cookies, tracking cookies or just cookies, are parcels of text sent by a server to a web client (usually a browser ) and then sent back unchanged by the client each time it accesses that server. HTTP cookies are used for authenticating , session tracking (state maintenance), and maintaining specific information about users, such as site preferences or the contents of their electronic shopping carts . The term "cookie" is derived from " magic cookie ," a well-known concept in UNIX computing which inspired both the idea and the name of HTTP cookies.
Cookies have been of concern for Internet privacy , since they can be used for tracking browsing behavior. As a result, they have been subject to legislation in various countries such as the United States , as well as the European Union . Cookies have also been criticized because the identification of users they provide is not always accurate and because they could potentially be a target of network attackers. Some alternatives to cookies exist, but each has its own uses, advantages and drawbacks.
Cookies are also subject to a number of misconceptions, mostly based on the erroneous notion that they are computer programs . In fact, cookies are simple pieces of data unable to perform any operation by themselves. In particular, they are neither spyware nor viruses , despite the detection of cookies from certain sites by many anti-spyware products.
Most modern browsers allow users to decide whether to accept cookies, but rejection makes some websites unusable. For example, shopping carts implemented using cookies do not work if cookies are rejected.
Doesn’t rely on cookies
Requires data-capture and cooperation at both ends
Data loss does occur
Only works to a point:
Very good for tracking coupon response rates
No good for shopping carts
EXERCISE: NEW YORK TIMES
The New York Times Online is one of the most trafficked English language news sources
It is the most linked-to web site by English language blogs
Its news feeds are ‘pumped’ into search indexes of all the major search engines via an XML feed
Why does it need to buy key words?
How would you recommend that it measures success?
ONLINE MEDIA MEASUREMENT DATA IS NOT GOOD; AND YET ITS WORLD CLASS
Lots & lots of data
It’s all digital and networked
Can track directly to sales online
First medium to measure the ad (not just content)
Lots & lots of data
Do we have the systems to handle data
There is Fraud and manipulation
Consumer has control
BIG privacy issues
Huge discrepancies creates mis-trust
Main Terms of the Medium
A unique visitor is a statistic describing a unit of traffic to a Web site, counting each visitor only once in the time frame of the report. This statistic is relevant to site publishers and advertisers as a measure of a site's true audience size, equivalent to the term " Reach " used in other media.
The Unique Visitors statistic is most accurately measured in two ways with current technology:
by requiring all Visitors to log-in to the site, thereby capturing the identity of each Visitor on each visit, or
by placing a cookie on each Visitor's computer, writing the cookie ID to a database, and checking for the cookie on each Visitor's computer each time they visit.
A series of requests from the same uniquely identified client with a set timeout. A visit is expected to contain multiple hits (in log analysis) and page views.
A page view (PV) or page impression is a request to load a single page of an Internet site . On the World Wide Web a page request would result from a web surfer clicking on a link on another HTML page pointing to the page in question. This should be contrasted with a hit , which refers to a request for a file from a web server . There may therefore be many hits per page view.
Impressions or Ad Views
Same as Page Views, but for the advertisements. Defined as communication
Please never use the word “Hits”
ONLINE MEDIA RESEARCH OVERVIEW
GLOBAL GUIDELINES HIGHLIGHTS
Refined definitions and standards
Client side measurement
Via a beacon/clear gif or client side call (i.e., 302)
Spiders & Bot Filtering (database)
2 step process, via 1) short list of bots (20-25), 2) known browser
Behavioral Filtering to remove non human activity
Might not be relevant if we use 2 step process above
Do not exclude as it is insignificant
Agree to header based cache busting
GLOBAL GUIDELINES CONTINUED
Shared “Areas of Auditing”
Asked to Communicate Internal Control Best Practices
Goal is Transparency
Description of Measurement Methodology
Data Collection Methods
Editing, Data Adjustment, etc.
General Reporting Parameters
Certification and/or Auditing Applied
THE OLD WAY: SERVER-SIDE SERVING AND COUNTING A B C 1 : User requests content from publisher web server. 2 : Publisher web server calls Publisher Ad Engine to retrieve ads. 3 : The Publisher Ad Engine logs that it has served an ad. Publisher Ad engine returns an HTML blob to Publisher Web Server. Some of these ads may actually be pointers to a location on a Third Party server. 4 : The Publisher Web Server receives the HTML blob. 5 : The Publisher Web Server returns the page and the page begins to render on the user’s machine. 6 : While rendering the page, the browser determines that it needs to pick up an ad from a Third Party server. The browser fires off a separate thread to get the ad from the Third Party server. 7 : The Third Party server logs that it has served an ad. 8 : The Third Party server receives the request for the ad and returns a pointer to the location of the ad image by instructing the user’s browser to pick up the ad from an image server. 9 : The user’s browser makes a call to the image server where the creative resides. 10 : The Image server logs that it has served an image. 11 : The image server returns the image. 1 2 3 4 5 6 7 8 9 10 11 Publisher Web Server Publisher Ad Engine Publisher Ad Engine Log Third Party Ad Engine Log Image Server Log
Causes of discrepancies
Publisher count is higher
Publisher count is lower
Publisher count is higher
Filtering techniques may differ
Typically cause extreme discrepancies
THE BETTER WAY: CLIENT-SIDE SERVING AND COUNTING A B C 1 3 4 5 2 6 7 8 9 10 11 Publisher Web Server Publisher Ad Engine Publisher Ad Engine Log Third Party Ad Engine Log Image Server Log
THE WAY IT’S BEING DONE NOW: SERVER-SIDE SERVING WITH CLIENT-SIDE COUNTING A B C 1 2 3 4 5 6 7 8 9 10 11 Publisher Web Server Publisher Ad Engine Publisher Ad Engine Log Third Party Ad Engine Log Image Server Log D 5b Publisher Beacon Server Publisher Beacon Log 5a 5c
HOW GOOD IS NNR AND COMSCORE DATA?
NetRatings and comScore Page View and Unique Visitor data trend together (are positively correlated) for less than half of websites examined.
NetRatings and comScore trend together for both Page Views and Unique Visitors for only four of the nineteen sites
Overall, the lack of consistency between the two services is no worse (and no better) in the second half of 2006 than it was in late 2005
The average NetRatings/comScore monthly difference for Unique Visitors across the nineteen sites ranges from 15% to 25% over the 13-month period, with no particular trend. For Page Views the average monthly difference has settled around 40%
There is a tendency for a majority of individual websites to be significantly (and consistently) higher in either NetRatings or comScore.
In those cases where differences are significant, NetRatings and comScore are each higher half of the time
In most cases, the two services are not close, and are reporting different “realities” regarding usage of specific websites
Differences in websites where one service is consistently higher may be related to the demographic make-up of the panels
SAMPLE NY TIMES - UNIQUE VISITORS www.gregstuart.com
SAMPLE: NY TIMES - PAGE VIEWS www.gregstuart.com
Think about a transaction that involves multiple customer touches offline and online?
Visit to the website
How do you measure the performance of this interaction? What value do you assign to online from the new customer?