AllstateChanging How the World SeesDigital AdvertisingLINDA ABRAHAM Co-Founder, CMO & EVP of Global DevelopmentANNE HUNTER SVP, Advertising EffectivenessANDREA VOLLMAN Marketing DirectorvCE™ CHARTER STUDY REVEALS HOW VALIDATED ADS ARE IMPACTINGTHE FUTURE OF DIGITAL AND CROSS-MEDIA MEASUREMENTMARCH 2012
2ContentsIntroductionValidated Campaign MeasurementThe vCE Charter StudyExecutive Summary of Findings In-View Audience Geography Brand Safety FraudImplications: Putting all of the Pieces TogetherConclusionAbout comScore3457816212325283031
3Across the globe, digital media has become an importantcomponent of every advertiser’s marketing mix. Accordingto the Interactive Advertising Bureau (IAB), display-relatedadvertising spending in the United States (U.S.) reached $10billion in 2010 and has grown at 20%+ rates since then, farexceeding the growth of traditional media. Just as we’ve seentremendous growth in terms of the volume of digital advertising,the landscape itself has also experienced a massive evolution.From new ad formats and placement strategies to new deliverysystems and ad technology, it has become challenging forplayers across the industry to stay up-to-speed.Until now, digital advertisingmeasurement has not kept pace withthe complexity of these changes.The transactional focus has been onmeasurement of gross impressionsdelivered, as opposed to those that wereactually seen by consumers in a particulartarget. As a result, marketers have beenlimited in their ability to understand howonline advertising works, especially whencompared to other media channels.This lack of understanding has resultedin reluctance by many marketers tofully embrace digital advertising. Frompublishers to ad networks and frommarketers to agencies, key players in thespace are calling for more transparencyand greater accountability as it relates toonline ad delivery.Addressing this industry-wide call-to-action, the IAB, the American Associationof Advertising Agencies (4As) and theAssociation of National Advertisers (ANA)– each representing a key constituentgroup in the advertising market – jointlylaunched an initiative called, MakingMeasurement Make Sense (3MS). Simplyput, 3MS’s goal is to improve, standardizeand simplify digital media measurement.In order to reach this goal, 3MS haspublished guidelines and is conductingresearch to help address issuessurrounding ad delivery, measurementand validation.IntroductionStay ConnectedFollow @comscoreFOR FURTHER INFORMATION,PLEASE CONTACT:Andrea VollmancomScore, Inc.+1 212 497 email@example.com
4In January 2012, comScore released a breakthrough innovationto the marketplace that addresses many of the guidelinesoutlined in the 3MS initiative as well as some additionalindustry issues relating specifically to ad delivery validation.This solution, validated Campaign Essentials™ (vCE™), providesan unduplicated accounting of impressions delivered across avariety of dimensions, helping to significantly improve the valueof online advertising.Importantly, vCE gleans all this information via a single ad tag, thus enabling acomprehensive, but holistic, view of digital ad delivery that is unique to the marketplace.The use of a single ad tag is a critical component of this measurement approach as itevaluates all impressions consistently and applies validity conditions simultaneously. Thiseliminates all issues associated with duplicated measurement and offers a more accurateview of campaign delivery. Duplication and inconsistency typically arise when disparatedata-collection sources are merged, which can dramatically impact the quality of theanalyzed data.vCE validates whether or not impressions delivered aspart of a campaign were: In-view (i.e. viewable by an actual consumer) Delivered in the right geography Seen in brand safe environments Absent of fraud In addition, vCE evaluates the demographic and behavioral composition of thecampaign audience, enabling the advertiser to assess the degree to which validatedimpressions reached the desired campaign targets.Validated CampaignMeasurement
5To better understand issuesassociated with display addelivery and validation, and totest-drive vCE, twelve leadingmarketers participated ina U.S.-based charter study,called the vCE Charter Study.The eye-opening findings help to pavethe way for a more accurate measure ofcampaign delivery that relies on validatedimpressions, rather than served impressions(or gross impressions), which are currentlythe established currency for online admeasurement. Validated impressions canalso be used to report validated gross andtarget rating points (vGRP/vTRP).Ideally, this research will help to promote thebroad adoption of new standard measuresthat reflect the true delivery of a campaign(per the 3MS guidelines), and it will alsohelp to generate greater visibility andtransparency across the industry and acrossmedia. Throughout 2012, similar charterprograms will be rolled out in other globalmarkets, including Canada, Latin America,Asia and select European countries.Parameters and MethodologyStudy Participants:Importantly, 100% of the vCE CharterStudy ad impressions were delivered iniframes, including a majority of ‘cross-domain’ iframes. The definition of theseiframes is discussed in the In-View sectionof this paper, but it is important to note thatthis is the first industry studyto measure and report onin-view rates for ads deliveredvia all iframes, including thosedelivered via the notoriouslydifficult-to-measure cross-domain iframes.For the purposes of this report, allfindings are presented in aggregate, notby individual campaign, to protect theconfidentiality of client data. Findings arereported by total campaign as well as bypublisher-level, placement-level and/orcreative-level.It should also be noted that because vCECharter Study participants included majorbranded advertisers, who inherently buymore premium inventory than the averageonline marketer, the study findings are notnecessarily representative of the overallonline advertising market. In fact, becausethese advertisers generally engage in high-end, premium campaigns, the findings mayrepresent ‘best-case scenarios,’ rather thanthe norm.The vCE Charter StudyThis is the firststudy to bringtwelve leadingmarketerstogether toholisticallyunderstand howonline advertisingis delivered.Time Period: December 2011 Total Campaigns: 18Media Placements: 2,975Site Domains: 380,898Ad Impressions: 1.8 billionFormat: All ads were display, delivered via iframes.
6In-view: In-view is defined as an adimpression with at least 50% of the ad’spixels in the user’s viewport for one secondor more. This definition is consistent withcurrent working standards outlined aspart of the 3MS initiative. The parametersfor the definition of in-view can be easilychanged to accommodate any change inindustry standards.Audience: Using the comScore panel of2 million global consumers, comScore isuniquely qualified to report on audiencedelivery with person-level insights. Thismeans the study was able to validatedelivery to target audiences based ontraditional demographics as well as morethan 80 behavioral segments.Geography: Geographic validation ismeasured by country on a global basis.Although vCE is available globally, withregional data available in some countries,for the purposes of the vCE Charter Study,all campaigns were validated based ondelivery in the U.S.Brand Safety: Ads delivered on sitesdeemed not appropriate for brandadvertising due to objectionable contentare considered to be in violation of brandsafety. The definition of objectionablecontent is further discussed within theBrand Safety section of the paper.Fraud: Fraud was measured by countingad impressions served to non-humanagents as per the IAB spiders and botslist as well as ads that were served tousers via illegitimate methods or content.Although there are several other typesof fraud detections, these two very basictypes were included in the vCE CharterStudy to establish a baseline.The goal of the vCE CharterStudy was to quantify theincidence of sub-optimal addelivery across these keydimensions for the advertisedbrands, and in so doing, framethe relative importance ofeach for the industry. AlthoughvCE offers the ability tooptimize campaigns in-flight inorder to eliminate waste andgenerate better advertisingoutcomes, this feature was notdeployed for the purposes ofthe study, as it would detractfrom the study’s objectiveof determining a baselineof delivery prior to in-flightoptimization.The vCECharter StudyKey Metric Definitions
71 In-View Rates are Eye-OpeningThe study showed that 31% of ads werenot in-view, meaning they never had anopportunity to be seen. There was alsogreat variation across sites where thecampaigns ran, with in-view rates rangingfrom 7% to 100% on a given site. Thisvariance illustrates that even for majoradvertisers making premium buys, thereis a lot of room for improvement.2 Targeting Audiences BeyondDemos Can be PowerfulGenerally, campaigns that had very basicdemo targeting objectives performed wellwith regard to hitting those targets. Forexample, those with an objective of reachingpeople in a particular broad age range did sowith 70% of their impressions. Predictably,as additional demographic variables wereadded to the targeting criteria (i.e. income+ gender), accuracy rates of the ad deliverydeclined. However, the results also showedthat, on average, 36% of all impressions in acampaign were delivered to audiences withbehavioral profiles that were relevant to thebrand (i.e. consumers with demonstratedinterests in categories, such as food, autoor sports). One campaign had 67% of itsimpressions viewed by the target behavioralsegment, demonstrating that targeting topeople based on interests or behaviors holdsstrong potential.3 The Content In Which An AdRuns Can Make or Break a BrandOf the campaigns analyzed, 72% hadat least some impressions that weredelivered adjacent to objectionablecontent. While this did not translate toa large number of impressions on anabsolute basis (141,000 impressionsacross 980 domains), it is important tonote that 92,000 people were exposed tothese impressions. This demonstrates thateven with the most premium of executions,brand safety should be an utmost concernfor advertisers.4 Fraud is the Elephantin the Digital RoomFraud is an undeniably large and growingproblem in digital advertising. The resultsshowed that an average of 0.16% ofimpressions across all campaigns wasdelivered to non-human agents from theIAB spiders & bots list. Although thispercentage might appear negligible, thereare two important considerations to keepin mind. Only the most basic forms ofinappropriate delivery were addressedin this study. When additional, moresophisticated types of fraud are considered,the problem will only get larger. Like brandsafety, fraud should be an importantconcern for all advertisers.5 Digital Ad Economics: The GoodGuys Aren’t Necessarily WinningThe study showed that there was littleto no correlation between CPM andvalue being delivered to the advertiser.For example, ad placements with strongin-view rates are not getting higherCPMs than placements with low in-viewrates. Similarly, ads that are doing wellat delivering to a primary demographictarget are not receiving more valuethan those that are not. In other words,neither ad visibility nor the demographictarget delivery is currently reflected in theeconomics of digital advertising.Executive Summaryof Findings
9calls a third-party domain to serve the ad.This severed communication link presentsa daunting challenge to the measurementof the iframe’s position on the page, and,therefore, ad visibility.The vCE technology is uniquein the marketplace as it isthe first and only that cansee through cross-domain, orunfriendly, iframes, which meansthat vCE’s in-view rate accountsfor all ad delivery formats.This is particularly important given thatcomScore research shows that 61%of iframed ads are delivered via theseunfriendly iframes. To demonstrate thevalue of this patent-pending technology,100% of the ads served in the vCE CharterStudy were delivered via iframes.In-View by Campaign & SiteAcross all campaigns in the vCECharter Study, the average in-view ratewas 69% (See Figure 1). The in-viewrates by campaign, however, showedsignificant variation – with a range of55% to 93%. This indicates that, onaverage, 3 out of 10 ads were notseen and were therefore wasted.61%of iframed adsare deliveredvia cross-domainor unfriendlyiframes.12345678910111213141516171893%85%84%80%76%75%74%73%71%69%68%66%65%59%58%58%57%55%Figure 1 Percentage of Ads In-View by Campaigncampaign
10Figure 2 Percentage of Ads In-View by Site69%100%minimum average maximumA site-level view across campaignsrevealed even more variation in in-viewrates (See Figure 2). On one site, 100%of the ads were in-view. For this particularsite, all ads were placed in the centerof the homepage and scrolling was notrequired to reach the remainder of thecontent. For another site, only 7% of thedelivered ads were in-view, meaning 93%of all ads served on that site never had theopportunity to be seen and were thereforecompletely wasted.Since only one out of every 14 ads onthe site had the opportunity to be seen,if a marketer paid a $1.00 CPM to deliveradvertising on that site, the effectiveCPM would have been $14.00. While asite with a $1.00 CPM may seem like abargain, when waste levels on the site areas high as 93%, it can effectively becomeone of the most expensive placements ina media plan.7%
11To better understand in-view rates, the results were analyzed by:• Placement (premium, standard, etc.)• Relative Size of Site (overall and within category)• Content Type (News sites, Sports sites, etc.)• Ad Size (300x250, 728x90, 160x600)• Position on the Page (above-the-fold versus below-the-fold)In-View by PlacementEven within a given site, in-view rates can vary significantly by placement.A traditional content site, for example, ran several vCE Charter Study campaigns. Acrossthe various placement locations on this site, the in-view rate varied from 23% to 95%.The placements appeared to fall into three distinct levels of in-view (See Figure 3).• The largest number of placements delivered more than 80% of the advertisementsin-view—well above the vCE Charter Study average (69%). Such placements couldbe considered high-viewability inventory.• Approximately one-third of the placements delivered advertisements between 66% to75% in-view, which indicates they were on-par with the vCE Charter Study average.• A small number of placements, however, dragged down the site’s average, giventheir very low in-view rates. With the use of in-flight optimization (which was notdeployed for the purposes of the vCE Charter Study), these sub-par in-view ratescould have been identified early and removed from the delivery. In addition, thesedata suggest an opportunity for this publisher to reconfigure the page layout toensure that more advertisements are viewable.Figure 3 Percent of Ads Delivered In-View for Individual Placements Across a Traditional Content Site100%80%60%40%20%0%%ofadsin-viewlow average high
12In-View by Relative Size of SiteAn important question relating toviewability is how in-view rates vary basedon the size of a site. To begin to answerthis question, a separate grouping ofaverage in-view rates was created basedon site size. Using comScore MediaMetrix® rankings within specific contentcategories (i.e. Sports sites, News sites,Food sites, Health sites, etc.) as a proxyfor site size, average in-view rates werecalculated based on Top 50, Top 100, Top500 and long-tail sites by category, andthe findings were then analyzed. Withinthese content categories, in-view levelsdecreased as the site rankings decreased.In fact, the difference in in-view ratesbetween Top 50 sites versus the long-tail sites was a full 16-percentage points(See Figure 4).This finding suggests that large siteswithin a content category do a better jobthan smaller sites at ensuring the adsthey deliver to consumers are actuallyviewable. Further analysis is needed toidentify exactly why this is the case, but afew potential options may include the factthat the quality of the site and the contentwithin a site is stronger on these morepopular sites.The differencein in-view ratesbetweenTop 50 sitesversus the long-tail sites in theircategory was afull 16-percentagepoints.In-View by content TypeIn-view rates also showed variation bycontent type (See Figure 5). For example,Coupon sites delivered relatively strongin-view rates (89%), whereas Pet sites(27%) struggled, delivering slightlymore than a quarter of ads in-view. Thisvariation across categories might, in part,reflect the common layouts among sitesin a similar genre.Figure 4 Percentage of Ads Served In-View within a Given Site Category80%75%70%65%60%%ofadsin-view77%74%70%61%top 50sitestop 100sitestop 500siteslong tailsites(remaining501+ sites)
13% of ads in-viewFigure 5 Percent of Ads Served In-View by Select Content TypesCouponsdirectoriesentertainment - radioentertainment - tvtechnology - newsweatherteenspoliticsdepartment storespets89%81%73%72%70%67%60%59%58%27%In-View by Ad SizeThe most common ad size used in thevCE Charter Study was the ClassicLeaderboard (728x90), followed bythe Medium Rectangle (300x250),and then the Wide Skyscraper(160x600). The Classic Leaderboarddelivered the strongest in-view rates(74%), but there was significantvariance across all sites with a rangeof 7% to 93% using this size. TheMedium Rectangle format (300x250)delivered 69% of its ads in-view,and the Wide Skyscraper (160x600)delivered the lowest portion of adsin-view (66%).Although further research is requiredto better understand the drivingfactors for differing in-view ratesacross ad sizes, one potential causeis the relationship between ad sizesand their typical placement on a webpage. For example, Wide Skyscraperads run vertically along a web page,making it more difficult for 50% of itspixels to be in the user’s viewport forat least one second.Figure 6 Percent of AdsDelivered In-View by Ad Sizewideskyscraper(160x600)mediumrectangle(300x250)leaderboard(728x90)66%69%74%
14In-View by Position on PageWhen discussing viewability, there is acommon misperception that ads delivered‘above-the-fold’ are seen, while adsdelivered ‘below-the-fold’ are not. Whilethe quality of in-view rates can vary from‘above-the-fold’ versus ‘below the fold’ addelivery, the vCE Charter Study resultshelp to dispel some of these myths.Surprisingly, the findings demonstrate thatsome ads delivered ‘above-the-fold’ werenot seen because users quickly scrolledpast them before the ad had a chanceto load, and alternatively, many adsplaced ‘below-the-fold’ delivered a highopportunity to be seen (See Figure 7).The implications of these findingsare far-reaching, and there are broadapplications for both buyers and sellersof online media. Publishers, for example,should monetize all ads on their sitethat deliver an opportunity to be seen,regardless of where the ad is placed onthe site. This might mean that inventory‘below-the-fold’ can be priced as premiumas long as the publisher can prove it wasviewed. Alternatively, marketers can lookfor inventory that is currently identified asremnant, which still delivers attractive in-view rates. Much of this inventory residesin exchanges and can be better optimizedby taking into account its placement-specific viewability potential.Figure 7 Percentage of Ads In-View by Location on PageThere’s goldbelow the fold.Marketers andpublishers whocan determinewhat is in-view bypage location havean advantage.
15In-View & CostFinally, comScore explored the relationshipbetween the cost of the ad and the in-view rate. Eight of the vCE Charter Studycampaigns provided cost data for usein the analysis. Some campaigns werebranding-oriented, while others weredirect response. In total, 300 unique adplacements had accompanying CPM data.The analysis showed there is virtually nocorrelation between the CPM paid for thead and whether it was in-view (correlationcoefficient = 0.19). This low correlationclearly demonstrates that sites with theability to garner strong in-view rates arenot being compensated fairly. Withoutsolid in-view data, current pricing fails toaccount for differentials in in-viewrates. Understanding the actual deliveryby both site and placement is critical formarketers seeking to value media basedon its ability to reach a real user.Publishers and marketers withdetailed in-view data are betterable to value the placementsthat provide true value andprice them accordingly.Figure 8 Correlation of In-View Rates & CPMR = 0.03729-501001502002503000% 20% 40% 60% 80% 100% 120%CPMIndex% of Ads In-View3002502001501005000% 20% 40% 60% 80% 100%cpmindex% of ads in-viewR2= 0.0373
16Marketers invest in digital with the goal ofbuying ads that are more successful thantraditional media at reaching a desiredaudience. Unfortunately, the extent to whichan ad reaches its target can vary greatlybased on many factors. The comScore vCECharter Study evaluated audience deliveryin two separate, but important, ways:Traditional DemographicsDelivery of ad impressions to traditionaldemographic targets, including age,gender, household income and thepresence of children in the home.Behavioral SegmentsDelivery of ad impressions to behavioralsegments based on observed onlinebehaviors (i.e. food enthusiasts,sports fans, etc.).Validating ad delivery based on traditionaldemographics is the most commonapproach. However, understanding howwell an ad reached a relevant behavioraltarget is potentially more valuable, sinceit offers perspective on not just whothe person is but on what they areinterested in, especially as it relatesto the advertised product.To evaluate the accuracy of ad delivery, vCECharter Study participants identified theirtarget audiences for each campaign, whichcould include one or any combination of thetraditional demographic attributes as well asbehavioral segments. Behavioral segmentsare comprised of the heaviest consumers(top 50%) of topic-specific Web content(i.e. sports, food, cars, personal electronicsor travel). vCE Charter Study participantsidentified a primary behavioral attribute from80 different online behavioral profiles.Figure 9 illustrates the most popular desiredattributes across all campaigns in the vCECharter Study. The majority of campaignsincluded age in their target set, whichis not surprising given its wide use as adesired attribute across all forms of media.Interestingly, however, the ability to reacha behavioral attribute was the next mostcommon approach, demonstrating thegrowing importance of some marketers’desire to reach people based on more thandemographics. It should be noted that theuse of behavioral campaign reporting isrelatively unique to digital and certainly acompelling value proposition for marketerstrying to connect more closely withconsumers who exhibit interests thatare aligned with and/or related to theadvertised brand.AudienceDefining Target Audienceage94%behavioralsegment50%gender44%householdincome6%presenceof child6%Figure 9 Percent of Charter CampaignsUsing Desired Attribute(s)
17Audience Targeting by Traditional DemographicsAcross all vCE Charter Study campaigns, there was quite a bit of variance in theirability to reach the desired target audience. As one might imagine, the more complexthe target (i.e. the more demographic targeting variables included in the target set),the more difficult it was for the campaign to deliver on its promise (See Figure 10).Campaigns with a target audience that included one demographic variable(e.g. 25-54 years old) delivered impressions to the target an average of 70% ofthe time. In cases where there were two variables (e.g. women + 25-54 years old),the accuracy of targeting decreased to an average of 48%, and with three variables(e.g. women + 25-54 years old + with children under 18 in the home) the averagewas 11%.Figure 10 Percent of Ads in Demographic Target Based on Number of Targeting Variables**Demographic variables can include: age, gender, household income and/or number of children in the household. Due tosample size, a meaningful range could not be calculated for campaigns with 3 demographic variables.100%80%60%40%20%0%%ofAdsDeliveredtoDemographicTarget1 variable70%2 variables48%3 variables11%LOW average high
18Audience Targeting byBehavioral AttributesIn addition to looking at the audiencesin terms of their demographics, onlinebehaviors of people who were exposedto the campaign were also measured.The campaigns were measured againsttheir desired behavioral attribute at thecampaign level. In some cases, specificcookie-based behavioral targetingwas used in several placements in thecampaign. In other cases, marketerswanted to reach their desired behavioralaudience through traditional mediaplacements, such as delivering an adalongside content of interest to theiraudience. Across all campaigns, theaverage campaign reached its behavioralaudience target 36% of the time, with awide range from 23% to 67% (SeeFigure 11).One obvious conclusion from this findingmay be that online behavioral targetinghas limitations as an accurate or effectivemeans of reaching audiences online.However, if executed correctly, behavioraltargeting can be a very powerful, efficientand effective means of delivering abrand message to a valuable audience.One primary reason for these limitationsincludes the cookie-based nature ofbehavioral segmentation. For example,while a user may have visited a travelsite that shared its information with dataproviders on the basis of the cookie forthat browser/machine combination, thereis no guarantee that when that cookie isobserved later at some other site, that itrepresents the same user. Another reasonrelates to the freshness of the information.Someone may have visited a travel sitesix weeks ago, but they are no longeractive in travel research. Finally, one visitalone may not be sufficient to identify aserious travel intender. As a result, onemust be careful about the accuracy of thetargets they purchase, which is preciselywhy audience validation and in-flightoptimization should be a critical part ofthe campaign management process. Ifthese campaigns were to have leveragedin-flight optimization (which they explicitlydidn’t for the purposes of the research),it is likely that these numbers would bedramatically higher.Figure 11 Percent of Ads Delivered to the Primary Behavioral Attribute by Campaign23%LOW36%average67%HIGH
19It is also important to note that, in somecampaigns, the behavioral attributetarget actually did a much better job atdelivering on-target impressions thanthe demographic group, suggesting thatusing demographics alone to evaluatethe success of campaign delivery is notsufficient. For example, one campaign fora CPG-product that had a demographictarget of women between the ages of25-54 years old, only served 37% ofimpressions to that group. However 67%of the impressions went to people whowere heavy users of food and cookingcontent online. With demographic-basedevaluation alone, this campaign deliverywould have appeared unsuccessful.A separate analysis of an Automotivecampaign in the vCE Charter Study helpsto shed light on the value of behavioralcampaign reporting and its ability to reveala deeper portrait of the type of consumerexposed to the campaign. The analysisinvolved creating an index of visitationto online site categories for consumersexposed to the ad campaign comparedto the average Internet population. Thefindings revealed that the exposedgroup over-indexed (158) on automotivecontent, meaning the audience reachedby the campaign was 58% more likelythan the average Internet user to be asignificant consumer of online automotivecontent (See Figure 12). This is a positiveindication that the campaign reachedthe right audience regardless of thedemographic composition.Another important finding was that theaudience reached in this campaign alsoover-indexed significantly in categoriesrelating to Financial Products (382)and Family and Parenting (266). Thisinformation can be used to developcreative messaging that speaks tothe interests of the audience, such asshowcasing a family vehicle or financinginformation in ads.Again, it is important to note that forthe purposes of the vCE Charter Study,these campaigns were not optimizedin-flight, meaning that no corrective actionwas taken throughout the course of thecampaign to improve the extent to whichthese ads were able to reach their targetaudience. With in-flight optimization,it is highly likely that all campaigns wouldhave seen improved on-target deliveryrates for both their demographic andbehavioral targets.Figure 12 Index of Online Behavioral Activity by Category for Consumers Exposed to an Automotive CampaignUsingdemographicsalone to evaluatecampaign deliverymay notbe sufficient.biz - financial productsfamily and parentingauctionsgaming - onlinegamblingautomotivesports and recreationteensshopping - fashionshopping - toysjobs - seekerspersonals382266262249227158141109453573index0 100 200 300 400
20Audience Targeting & CostUsing available CPM data (as outlinedin the prior In-View section), thecorrelation between CPMs and theaccuracy of demographic targeting(primary audience only) was analyzedas part of this research. The findingsrevealed a very small correlation(correlation coefficient = 0.18),suggesting that there is little or norelationship between the amount paidfor an ad and its ability to reach thedesired demographic target audience.Before drawing macro conclusions aboutthis finding, it is important to examinesome of the potential reasons for thislack of correlation between these twovariables. First, some marketers mightsimply not be building campaigns withthe core objective of reaching a specificdemographic, and instead they are buyingmedia based on its ability to hit certainbehavioral segments.Another very real issue is the accuracy ofcookie-based targeting data. As notedabove, there are a myriad of companiesthat provide this data, and there isvery large variation in the quality of thedata. Unless cookie-based audiencesare verified against a credible third-party source, it is possible that they aremissing the mark. In the vCE CharterStudy, demographically-cookie targetedad placements reach their desireddemographic 14% to 96% of the time.This indicates a wide variation on thequality of demographic cookie data.Regardless of the cause, it is clear that,at present, the market is not rewardingads that deliver to the intended audiencecompared to those that did not. Thisrepresents an opportunity for bothadvertisers and publishers, especiallynow that they have transparency into theaccuracy of delivery and the ability tooptimize in-flight to avoid waste.Figure 13 Correlation of % of Ads Delivered to Primary Demographic Target & CPMUnless cookie-based audiencesare verified againsta credible,third-party source,it is possible thatthey are missingthe mark.0501001502002503000% 20% 40% 60% 80% 100% 120%IndexofCPM% of Ads Delivered to the Primary Demographic Target3002502001501005000% 20% 40% 60% 80% 100%indexofcpm% of Ads Delivered to the Primary Demographic TargetR2= 0.0308
21india philippines australia germany s.korea dom.rep. indonesia brazil franceU.K.Figure 14 Percent of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of North America%ofimpressions1%0%2%3%4%5%6%7%8%9%Defining Geographic TargetWhen delivering ads on television, it’srelatively easy to ensure they run in theirdesired geographic market, becausebroadcast markets have defined geographicborders. The Internet, on the other hand, isborderless and users can access specificcontent from anywhere in the world. As aresult, controlling geographic distribution ofadvertising can be challenging. For marketerstrying to maximize every dollar of theiradvertising budget, it is critical that their adsare delivered in the desired market wheretheir products are actually sold. Accordingly,geographic validation was an importantcomponent of the vCE Charter Study.Geographic Targeting:Overall & By CampaignAll campaigns in the vCE Charter Studyhad a geographic target of the U.S.,and in total, about 4% of impressionswere delivered outside of the U.S.Of impressions delivered outside ofthe target, nearly half were served inCanada, and the remainder spreadacross Europe, The Caribbean,Asia-Pacific and Latin America. Thisfinding suggests that a good portion ofthe wasted impressions were deliveredto people living in countries whosenative language is something otherthan English.Geography
22When examining the results on acampaign-by-campaign basis, it isinteresting to note the large range ofimpressions delivered outside of thetarget geography. While one campaignperformed flawlessly, another wastedabout 15% of its impressions (See Figure15). Given that the Internet providesa wealth of geo-location information,and given the campaigns’ broad targetof ‘inside the U.S.’, this large range issomewhat surprising.In such cases, the inability for an ad tobe delivered in its intended geographyis often not a result of poor targetingcapabilities, but rather due to errorin complex ad buying and sellingprocesses. Delivery of ads outside a givengeographic target often occurs for twoprimary reasons:The first reasonis simplecommunication error. In some cases, the siteserving the ad is not aware of the intendedgeographic target. This occurs when therequirement does not appear on the insertionorder (IO), which authorizes the purchase ofimpressions from the site and determines thecharacteristics of the ads to be served. Suchmisfires can be easily remedied by ensuringgeographic requirements are a standard partof IO contract templates.The second reason is due to humanerror. To target an ad to a given geography,the requirement must be programmedin the ad server that is delivering the ad.Occasionally this step is missed by thepublisher, or in rare cases, the wronggeography is inadvertently selected.Fortunately, there are easy ways to combatthese issues. As long as geography isspecified in the IO, performance can beoptimized in-flight in two different ways.The first is through real-time alerting, whichnotifies sites when ads are being servedoutside the desired geographic region sothat corrective action can be taken. Thesecond option is to use an ad blockingtechnology, which can be implementedto prevent ads from being served outsidethe geographic target altogether. This isgenerally reserved for instances whereserving an ad outside a specified geographymay create privacy or legal concerns, and inlieu of in-flight course correction, absoluteprevention must be employed. These alertand blocking features can protect bothmarketers and publishers from wastinginventory and from lowering the overalleffect of a campaign. Although neitheralerting nor blocking was used in the vCECharter Study, both of these features arepart of the comScore vCE offering.Figure 15 Percent of Ads Delivered In Geography by CampaignThe inability for anad to be deliveredin its intendedgeography isoften not a resultof poor targetingcapabilities, butrather due to errorin complex adbuying and sellingprocesses.75% 80% 85% 90% 95% 100%% of Ads in geographic Target123456789101112131415161718campaign
23A major concern of all marketers is therelevance of the content in which theirads are delivered. When brands spendmoney on advertising, they need assurancethat their ads will not run next to contentthat is at odds with the brand they aretrying to build or the equity they havealready established.In this context, ‘objectionable content’ cangenerally be categorized into two buckets,the first being rather objective and thesecond being much more subjectiveand brand-specific:Type I:Adult-Contentand/or Hate SitesAlmost all brands want to avoid having theirads run on Adult-Content or Hate sites.Although there might be some differencesof opinion on exactly what sites fall intothese categories, there are generally agreedupon and industry-endorsed lists thatdefine these, and almost without exception,reputable marketers want to avoid themat all costs.Type II:Brand-Specific CriteriaThere are topics, issues and/or contentthat certain brands don’t want to advertisenear because it directly conflicts withand/or detracts from the advertising’sobjective. For example, consider a majorairline. For obvious reasons, an advertiserin this space might not want the brand’sad to appear next to an article aboutsignificant plane delays. Meanwhile, forcountless other advertisers, deliveringan ad to a consumer in this contentwould be completely benign.Concerns relating to both of thesecategories are very legitimate.Unfortunately, though, due to the complexchain of online ad delivery through adnetworks and exchanges, it is not alwaysclear where an ad will appear.Brand Safety on Adult-Contentand Hate SitesTo begin to understand the extent towhich ads are delivered in contentdeemed inappropriate, the vCE CharterStudy quantified the incidence of addelivery on Adult-Content and/or Hatesites (Type I). The study used a standarddefinition of ‘objectionable content’, basedon historical data of sites/categories mostcommonly identified as being ‘not brandsafe’ by leading advertisers (See Figure16). The measurement was applied toall campaigns.Figure 16 Categories Deemed “Not Brand Safe”for Purposes of vCE Charter Study• Piracy andCopyright Theft• Anonymizer• Child AbuseImages• Criminal Skills• Hacking• Illegal Drugs• Marijuana• Spam URLs• Botnet• CommandControl Centers• Comprised andLinks to Malware• MalwareCall-Home• MalwareDistribution Point• Phishing/Fraud• Spyware andQuestionableSoftware• Peer-to-Peer• TorrentRepository• Hate Speech• Pay to Surf• Nudity• Pornography• Sex and Erotic• Content Server• Private IPAddress• RedirectBrand SafetyDefining Brand SafetyDue to thecomplex chainof online addelivery throughad networks andexchanges, it isnot always clearwhere an ad willappear.
24To the surprise of many advertisers in thevCE Charter Study, 72% of the campaignshad at least some impressions served inthis type of inappropriate content, whichspanned a total of 980 sites (See Figure17). The good news is that the actualpercentage of impressions involved wasquite small, less than .01%. However, thestudy also showed that 92,000 people sawthese ads, meaning that if some of thesepeople were either loyal or prospectivecustomers, it could be counter-productiveand/or problematic for the brand.It should be noted that it is likely that thisnumber is much higher when evaluatingthe broader, online advertising universeas there are certain factors that may havepositively influenced the low percentageof inappropriate ad placements in the vCECharter Study. These factors include:• The brands under measurement werepremium national marketers and thereforemore likely to use higher quality content• Many of the brands were already employingad blocking technologies from externalthird-parties. Even with these technologiesin place, several instances of inappropriateplacements still appeared.• In a few instances, select demand-sideplatforms chose to obfuscate the URLswhere the ads were run, meaning thatbrand safety could not be measured andclients could not validate where the adswere run.Despite the relatively low overall incidenceof ads appearing next to inappropriatecontent, these findings still might beunsettling to advertisers. Even onead impression delivered in the wrongenvironment can damage a valuableconsumer’s feelings toward a brand.With the increasing use of social media,a snapshot of a marketer’s ad in aninappropriate environment can quickly goviral, exposing many more people to theunintended, but negative, association ofa brand and inappropriate content. With92,000 people being exposed acrossall vCE Charter Study campaigns, theadvertisers’ concerns are justified.The daily alerts and blocking technologydiscussed in the geography section of thereport can also be deployed for Type 1 and/or Type 2 content sites. Real-time alerts canbe set to notify publishers, marketers and/or agencies if an ad is appearing in contentdeemed ‘not brand safe.’ In addition, thetechnology can completely block the adfrom being served in certain environments.The definition of what is brand safe can becustomized by the brand.Figure 17 Percent of vCE Charter Study Campaigns with Impressions Delivered Next to Content Deemed “Not Brand Safe”Some Ads in InapproPRiate contentno ads in InapproPRiate content28%72%72%of the campaignshad at least someimpressionsserved ininappropriatecontent, whichspanned a total of980 sites.92,000 peopleexposed tOAdult-Contentand/or Hate Sites
25Defining FraudToday’s world of online advertisinginvolves many players in the ecosystem,each with a specific role and goal.However, the inherent complexity in thislandscape results in a lack of control andvisibility into online ad delivery. While thevast majority of individuals in the digitaladvertising ecosystem operate with thebest of intentions, like any industry, thereare fraudulent players that can disruptthe value chain. The complicated daisychain of ad delivery can involve up to 20different players, and quite often neitherthe buyer nor the seller has insight intoeach step in the process.The term ‘fraud’ as it relates to onlineadvertising encompasses a variety ofimpression-delivery scenarios. In somecases, there is direct fraud, which isdeliberate and completely illegitimate,while other types of fraud are anunintentional by-product of legitimatebusiness practices. In either case, thisfraudulent activity does not deliver adsto actual people as intended, so shouldtherefore be excluded from validatedimpression counts.The vCE Charter Studyspecifically measured twoaspects of inappropriatedelivery:1. The incidence of ad deliveryvia non-human spiders andbots identified by the IAB2. The incidence of addelivery on sites withclear illegitimate andintentional fraudFraudThe complicateddaisy chain ofad delivery caninvolve up to 20different players,and quite oftenneither the buyernor the sellerhas insight intoeach step inthe process.
26List of Non-Human Spiders & Bots Identified by the IABTo help members of the online advertising ecosystem better understand and avoidissues relating to fraud, the IAB maintains a list of all known non-human spiders andbots. All IAB-accredited ad servers are required to filter out these known sourcesof non-human ad impressions. The use of some of the spiders and bots on this listis a completely legitimate practice employed by many websites for a variety of uses,such as to gather data to help index pages for search engines or to determine pagecontent for the purposes of offering contextual ad placements. Regardless of their use,however, they do not deliver ads to consumers and can therefore wreak havoc on addelivery and validation, causing a lot of wasted ad impressions and skewing the resultsof advertising effectiveness measurement. An analysis of vCE Charter Study campaignsshowed that the average campaign in the study had 0.16% of total impressions beingdelivered via these spiders and bots, with a range of 0.03% to 0.49% (See Figure 18).Figure 18 Percent of Total Impressions Delivered Via Non-Human Spiders and Bots as per IAB List1234567891011121314151617180.20%0.26%0.27%0.06%0.21%0.09%0.07%0.12%0.17%0.10%0.10%0.09%0.03%0.49%0.04%0.04%0.40%0.10%campaign
27Sites with Intentionally Fraudulent & Illegitimate ActivityIn addition to known spiders and bots, part of the vCE Charter Study analysis includedan evaluation of fraudulent impressions that were intentionally delivered via illegitimateonline activity. Campaign delivery was manually reviewed for unusual activity indicative ofintentional fraud. Such indicators might be unusually high or unusually low in-view ratesor little or excessive mouse movement on the creative. Upon identifying these outliers,further human investigation was used to either confirm or negate the hypothesis.The analysis revealed more than 200 sites that were guilty of this type of fraudulentdelivery. Figure 19 below outlines some of the most common categories of sites withsuch activity. Additionally, the investigation uncovered that one of the sites deliveredalmost two million ads in the vCE Charter Study, supporting the need for consistenthygiene on campaigns to accurately measure delivery and ensure only ads that aredelivered to actual humans are counted in validation and effectiveness measurement.Again these ads were not blocked from serving for the purposes of this study butinstances of delivery were measured.PhisHing/FraudRedirectmiscellaneouspay to surfpeer-to-peercriminal skillshackingcompromised & Links to malwaremalware distribution pointno content foundunreachablespam urlsanonymizertorrent repositorypiracy & copyright theftFigure 19 Custom Categorization of Sites with Intentionally Fraudulent Activity0 5 10 15 20 25 30 35 40Number of Websites WITHIN EACH CATEGORYWhile these two categories of fraudulent ad delivery accounted for only a smallpercentage of total ad impressions in the vCE Charter Study, there are a variety ofother sources of fraud that consistently result in significant waste. For perspective,of the approximately 1 trillion URLs that comScore processes each month(40% more than all the traffic of the entire U.S. Internet population),the application of comScore’s full suite of fraud detectiontechnologies identified levels of fraud ranging from 3%to 10% for a given campaign. Clearly, no brand is immune from fraud,and it should be an area of concern for all players in the ecosystem.No brand isimmune fromfraud, and itshould be anarea of concernfor all players inthe ecosystem.
28The vCE Charter Study demonstrates that each dimension of addelivery – viewability, audience targeting, geographic targeting,brand safety and fraud – has a significant impact on whether ornot an ad has an opportunity to achieve its intended objective,and should therefore be a central component of ad deliveryvalidation measurement.Advertisers want to understand addelivery to each of these core dimensions,and they also require a holistic, un-duplicated view of total campaign delivery.In order to achieve this un-duplicatedaccounting of delivered impressions,advertisers require a simple solutionthat eliminates all of the wasted time anderror associated with merging disparatedata sources. Consider, for example,results from a single campaign in thevCE Charter Study.IN ONE CAMPAIGN WHENMEASURED INDIVIDUALLY, THEFINDINGS SHOWED THAT:38.9% of the ads weredelivered to the righttarget audience58.0% of the adswere delivered in-view85.7% of the adswere delivered inthe right geographyBecause of duplication across thesethree dimensions, one cannot simply sumthe percentages, as this would suggestthat 155.9% of the ads were deliveredaccording to plan or that 118.4% of thead impressions didn’t deliver well. Instead,through the use of a single ad tag and asingle measurement solution, vCE is ableto validate that a combined total of33% of the ads were deliveredaccording to plan (See Figure 20).Implications: Puttingall of the Pieces Togetherin geography84.7%validTarget33%in-view58.0%ontarget38.9%Figure 20 Intersection of Percent of In-View,In Geography and On Target Ad ImpressionsDelivered For a Sample Campaign in vCECharter Study
29Prior to the introduction of vCE, thetechnology to validate all campaignimpressions against core criteria was notfully available. The vCE Charter Studydemonstrates that the technology nowexists to identify and correct the sourceof sub-optimal performance, and that theopportunity to do so is substantial. In fact, in aperfect world, advertisers and publishers cancontract and pay on the basis of impressionsthat were served for the campaign, but havealso fully met the validity criteria.vGRP: A Truly Cross-MediaComparable MetricIn order for marketers to plan, measureand evaluate media across channels,they require digital campaign deliverymeasurement that can be translated intotraditional metrics, like reach, frequencyand gross rating points (GRPs). A centralcomponent of vCE is the validated GRP,or vGRP. The vGRP provides the industrywith a cross-media comparable GRP metricthat is also meaningful in the context of howonline advertising works.vGRPs are calculated by removing allad impressions that did not have theopportunity to make an impact, includingthose that were not in-view, delivered to thewrong geography, served near brand unsafecontent and subject to fraud. Similarly,validated target rating points, or vTRPs,include an overlay of audience-validateddata, providing yet another actionable metricfor marketers seeking to plan campaignsacross channels.The example below of a CPG brand helpsto illustrate how vGRPs can impact thetrue reach and frequency of a campaign(See Figure 21). In this example, usingnon-validated impressions, the campaignappears to have delivered 46.7 GRPs.When using validated impressions,however, the campaign delivered 20.7vGRPs, yielding a vRatio of 44%. Thisdelta between GRPs and vGRPs in digitalmedia demonstrates the volume of wasteoccurring, and highlights significant areasfor improvement.IMPLICATIONSPutting all of thePieces TogetherFigure 21 Gross and validated GRP for a Sample Campaign in vCE Charter StudyGROSS VALIDATED RATIOReach 8.7 4.9 56%Frequency 5.4 4.2 79%GRP 46.7 20.7 44%TRP 61.4 24.5 40%
30While the vCE Charter Study sheds light across every aspectof delivery, three consistent themes emerged in the findings.1 Marketers are not necessarily getting what they expectwhen they buy online ads. From ads delivered next to objectionable content toads that never had the opportunity to be seen, there are countless examples where thedigital medium is simply not delivering on its promise.2 The way online advertising is delivered varies significantly bysite, placement and even creative. Across all dimensions of ad delivery, thevCE Charter Study demonstrated clear examples of situations where ad impressionswere largely wasted. These findings suggest that measuring all dimensions of addelivery for every placement in a holistic fashion is critically important.3 Regardless of the quality of the buy, there is almost alwaysroom for improvement. Advertisers who understand and leverage the power ofvalidation stand to gain much more value from the digital channel.The digital medium has advanced the discipline of advertising in many respects, but ithas also introduced significant complexity to the media equation. To maximize the valueof this important medium, it is important to have the tools to ensure the industry regainsits footing on some of the aforementioned pitfalls and continues to advance forward.The vCE Charter Study has illuminated many of the ways value is currently being left onthe table. Now is the time for advertisers, publishers and other industry stakeholders torealize that value.Conclusion: vCE CharterStudy Key ThemesvCE CHARTERSTUDY KEYTHEMES
31comScore, Inc. (NASDAQ: SCOR) is a global leaderin measuring the digital world and preferred source ofdigital business analytics. comScore helps its clientsbetter understand, leverage and profit from the rapidlyevolving digital marketing landscape by providing data,analytics and on-demand software solutions for themeasurement of online ads and audiences, mediaplanning, website analytics, advertising effectiveness,copy-testing, social media, search, video, mobile,cross-media, e-commerce, and a broad variety ofemerging forms of digital consumer behavior.comScore services, which now include the product suitesof recent acquisitions AdXpose, Nedstat, Nexius XPLORE,ARSGroup and Certifica, are used by more than 1,800clients around the world, including global leaders suchas AOL, Baidu, BBC, Best Buy, Carat, Deutsche Bank,ESPN, France Telecom, Financial Times, Fox, Microsoft,MediaCorp, Nestle, Starcom, Terra Networks, UniversalMcCann, Verizon Services Group, ViaMichelin and Yahoo!.For more information, please visit www.comscore.comAbout comScoreStay ConnectedFollow @comscoreFOR FURTHER INFORMATION,PLEASE CONTACT:Andrea VollmancomScore, Inc.+1 212 497 firstname.lastname@example.org