Next-Generation Campaign
  Section Title
 Validation & Optimization:
     A Sprint Case Study



     Victor Silva   Andrea Molette
In 2007, comScore’s First Post-Buy Analysis Across 8 Digital US
    Campaigns Showed Execution Left a Lot to be Desired …

      70%
                          Percent of Ad Impressions for 8 Campaigns
      60%


      50%


      40%


      30%           61%

      20%


      10%                                               19%
                                        8%                              12%
      0%
            In US But Not Target     Hit Target       Hit Target      Outside US
                                   Frequency >=5    Frequency <=4
Branding advertisers on TV are accustomed to audience
       guarantees and expect the same in digital
      Accuracy of cookie-based digital plan delivery is
                        problematic:
       Cookie Deletion                                Cookie Proliferation




           X             Cookies Are Not People




                                       Source: comScore 2011
Some Things We’ve Learned About Digital
         Media Plan Delivery
• The negative impact of cookie deletion
   – Cookie deletion inflates ad frequency and deflates ad reach
     by as much as a factor of 2.5X
• Targeting accuracy using cookies:
   –   70% for 1 demo (e.g. women)
   –   48% for 2 demos (e.g. women age 18-34)
   –   11% for 3 demos (e.g. women age 18-34 with kids)
   –   36% for behavioral targeting (e.g. people visiting travel sites)
Cookies Can’t Accurately Identify Who is Using a Computer at any
           Given Point in Time due to Multiple Users


         Over 64% of home users share a computer with other users




                          3+ users         1 user

                          32% of the36%
                               30%
                          time, someone other
                            than the Facebook
                            logged-in person is
                             actually using the
                                  2 users
                                 computer
                                 32%
Cookie Deletion is a Global Reality
             …and a Global Challenge
           Accurately counting reach with cookies is not possible,
yet is currently the method used in most ad servers and analytics systems

                                          Ad Server Cookies


                                Percent of         Average # of cookies
                Country      computers deleting   per computer for same
                                                        campaign


       Australia                    37%                       5.7
       Brazil                       40%                       6.6
       U.K.                         35%                       5.9
       U.S.                         35%                       5.4
Arguably the Most Important Digital Advertising Initiative To Date:
      Making Measurement Make Sense (3MS) Mission




    Reduce costs of doing business due to complexity of
     digital advertising ecosystem
    ‘Single Tag’ solution to reduce complexity
    Improve reporting of ad exposure
    Bolster confidence that ads delivered are actually
    visible
What is a vGRP?
• validated Gross Rating Points (GRP) based on validated
  impressions delivered to the Total Census Population for selected
  Geographic Market, or 100 * % Pop Reach * Average Frequency
  for the reporting period
• Comparable to GRPs used in television because it used the same
  calculations
• vGRPs must:
   –   Deliver in the target geography
   –   Not be fraudulent deliveries
   –   Be in brand safe content
   –   Be viewable
   –   Hit the target audience
vGRP can provide more accurate analysis of campaign
             effectiveness in Marketing Mix Models


       GRP: Negative                             vGRP: Positive
        Correlation                                Correlation
100                                   80   100                            80

 90                                        90
                                      70                                  70
 80                                        80
                                      60                                  60
 70                                        70                     vGRP
                              GRP     50                                  50
 60                                        60                     Sales
                              Sales
 50                                   40   50                             40

 40                                        40
                                      30                                  30
 30                                        30
                                      20                                  20
 20                                        20
                                      10                                  10
 10                                        10

  0                                   0      0                            0

        Month 1     Month 2                      Month 1   Month 2
CHANGING HOW THE WORLD SEES
    DIGITAL ADVERTISING
vCE US Charter Study:
12 Major Branded Advertisers Came Together to Lead & Learn

  18 campaigns
  2 billion impressions
  400,000 sites




                         Allstate
Charter Study replicated in Europe with similar results

  15 campaigns
  640 million impressions
  213,000 sites
Study Objective

Quantify incidence of sub-optimal ad delivery across key
                ad delivery dimensions…




… to better understand sources of waste and identify
opportunities to extract more value for all players in the online
advertising ecosystem
Importantly, all impressions in the study were
delivered in iframes, including the notoriously difficult-
          to-measure cross-domain iframes

    Directory on site   Same-domain iframe     Cross-domain iframe
                             (friendly)            (un-friendly)

         ad                   ad                         ad
         web                   web                      web
         site                  site                     site




                                          61+% of
                                       iframed ads
                                          use this
                                          method
Across all campaigns, the average in-view rate was
   69%, meaning 3 out of 10 ads weren’t seen

           Percentage of Ads In-View by Campaign
vCE Charter Study:
In-View Rates Need to Be Improved




 US                                EU
 69%                               67%
AVERAGE                           AVERAGE



Campaign In-View ad rates ranged from:
US 55% to 93% EU 64% to 72%
In-view rates by site ranged from 7% to
100%, suggesting the importance of validation across all
                         sites
   Percentage of Ads In-View by Site
Large sites scored better than long-tail sites

              Percentage of Ads Served In-View



    77%         74%                  70%
                                                 61%
    66%         63%                  61%
                                                 55%


                             US     EU
The Classic Leaderboard delivered the strongest in-view rates but
  there was significant variance across all sites with a range of
                    7% to 93% using this size
 Percent of Ads Delivered In-View by Ad Size




                                                      one potential
                                                            cause?
                                               the relationship between ad
                                                      sizes and their typical
                                                 placement on a web page
Digital Ad Economics:
     The Good Guys Aren’t Necessarily Winning

                Low correlation of In-View Rates & CPM


                                                                 R²=0.0373




An equally as weak correlation was also observed between CPM and ability to
                     hit a primary demographic target
On a campaign-by-campaign basis, one performed
flawlessly, while another delivered 15% of its impressions to
                    the wrong geography
                Percent of Ads Delivered In Geography by Campaign




   Sub-optimal geographic delivery is often a result of communication or human
                          error, and it can be remedied
                       with in-flight alerting and blocking
Of those ads delivered outside of their target, a good
portion of them landed in countries where English is
            not even the primary language
% of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of N.A.
Brand safety should be of the utmost importance to advertisers.
Even one poorly delivered ad can leave a very bad impression

Percent of Campaigns with Impressions Delivered
Next to Content Deemed “Not Brand Safe”
The Above-the-Fold Myth?
Above-the-fold in-view rates ranged from 48% to 100%




                                                 source: comScore vCE charter study
Some Below-the-Fold ads
are actually premium inventory
    Below-the-fold in-view ranged from 3% to 67%.




                                                    source: comScore vCE charter study
There is good news for advertisers and publishers

 Analogous to TV audience guarantees
 Eliminating unseen online inventory supply results
  in:
    More effective / efficient campaigns and less waste for
     advertisers
    More accurate metrics for market mix models
    Better proof of digital ad effectiveness
 Increased transparency/accountability means
  increased confidence in digital
Thank You!
Download the whitepaper, continue the conversation

Sprint's Next Generation Campaign Validation & Optimization

  • 1.
    Next-Generation Campaign Section Title Validation & Optimization: A Sprint Case Study Victor Silva Andrea Molette
  • 2.
    In 2007, comScore’sFirst Post-Buy Analysis Across 8 Digital US Campaigns Showed Execution Left a Lot to be Desired … 70% Percent of Ad Impressions for 8 Campaigns 60% 50% 40% 30% 61% 20% 10% 19% 8% 12% 0% In US But Not Target Hit Target Hit Target Outside US Frequency >=5 Frequency <=4
  • 3.
    Branding advertisers onTV are accustomed to audience guarantees and expect the same in digital Accuracy of cookie-based digital plan delivery is problematic: Cookie Deletion Cookie Proliferation X Cookies Are Not People Source: comScore 2011
  • 4.
    Some Things We’veLearned About Digital Media Plan Delivery • The negative impact of cookie deletion – Cookie deletion inflates ad frequency and deflates ad reach by as much as a factor of 2.5X • Targeting accuracy using cookies: – 70% for 1 demo (e.g. women) – 48% for 2 demos (e.g. women age 18-34) – 11% for 3 demos (e.g. women age 18-34 with kids) – 36% for behavioral targeting (e.g. people visiting travel sites)
  • 5.
    Cookies Can’t AccuratelyIdentify Who is Using a Computer at any Given Point in Time due to Multiple Users Over 64% of home users share a computer with other users 3+ users 1 user 32% of the36% 30% time, someone other than the Facebook logged-in person is actually using the 2 users computer 32%
  • 6.
    Cookie Deletion isa Global Reality …and a Global Challenge Accurately counting reach with cookies is not possible, yet is currently the method used in most ad servers and analytics systems Ad Server Cookies Percent of Average # of cookies Country computers deleting per computer for same campaign Australia 37% 5.7 Brazil 40% 6.6 U.K. 35% 5.9 U.S. 35% 5.4
  • 7.
    Arguably the MostImportant Digital Advertising Initiative To Date: Making Measurement Make Sense (3MS) Mission  Reduce costs of doing business due to complexity of digital advertising ecosystem  ‘Single Tag’ solution to reduce complexity  Improve reporting of ad exposure  Bolster confidence that ads delivered are actually visible
  • 8.
    What is avGRP? • validated Gross Rating Points (GRP) based on validated impressions delivered to the Total Census Population for selected Geographic Market, or 100 * % Pop Reach * Average Frequency for the reporting period • Comparable to GRPs used in television because it used the same calculations • vGRPs must: – Deliver in the target geography – Not be fraudulent deliveries – Be in brand safe content – Be viewable – Hit the target audience
  • 9.
    vGRP can providemore accurate analysis of campaign effectiveness in Marketing Mix Models GRP: Negative vGRP: Positive Correlation Correlation 100 80 100 80 90 90 70 70 80 80 60 60 70 70 vGRP GRP 50 50 60 60 Sales Sales 50 40 50 40 40 40 30 30 30 30 20 20 20 20 10 10 10 10 0 0 0 0 Month 1 Month 2 Month 1 Month 2
  • 10.
    CHANGING HOW THEWORLD SEES DIGITAL ADVERTISING
  • 11.
    vCE US CharterStudy: 12 Major Branded Advertisers Came Together to Lead & Learn 18 campaigns 2 billion impressions 400,000 sites Allstate
  • 12.
    Charter Study replicatedin Europe with similar results 15 campaigns 640 million impressions 213,000 sites
  • 13.
    Study Objective Quantify incidenceof sub-optimal ad delivery across key ad delivery dimensions… … to better understand sources of waste and identify opportunities to extract more value for all players in the online advertising ecosystem
  • 14.
    Importantly, all impressionsin the study were delivered in iframes, including the notoriously difficult- to-measure cross-domain iframes Directory on site Same-domain iframe Cross-domain iframe (friendly) (un-friendly) ad ad ad web web web site site site 61+% of iframed ads use this method
  • 15.
    Across all campaigns,the average in-view rate was 69%, meaning 3 out of 10 ads weren’t seen Percentage of Ads In-View by Campaign
  • 16.
    vCE Charter Study: In-ViewRates Need to Be Improved US EU 69% 67% AVERAGE AVERAGE Campaign In-View ad rates ranged from: US 55% to 93% EU 64% to 72%
  • 17.
    In-view rates bysite ranged from 7% to 100%, suggesting the importance of validation across all sites Percentage of Ads In-View by Site
  • 18.
    Large sites scoredbetter than long-tail sites Percentage of Ads Served In-View 77% 74% 70% 61% 66% 63% 61% 55% US EU
  • 19.
    The Classic Leaderboarddelivered the strongest in-view rates but there was significant variance across all sites with a range of 7% to 93% using this size Percent of Ads Delivered In-View by Ad Size one potential cause? the relationship between ad sizes and their typical placement on a web page
  • 20.
    Digital Ad Economics: The Good Guys Aren’t Necessarily Winning Low correlation of In-View Rates & CPM R²=0.0373 An equally as weak correlation was also observed between CPM and ability to hit a primary demographic target
  • 21.
    On a campaign-by-campaignbasis, one performed flawlessly, while another delivered 15% of its impressions to the wrong geography Percent of Ads Delivered In Geography by Campaign Sub-optimal geographic delivery is often a result of communication or human error, and it can be remedied with in-flight alerting and blocking
  • 22.
    Of those adsdelivered outside of their target, a good portion of them landed in countries where English is not even the primary language % of Ads Delivered to Geographic Market Among All Impressions Delivered Outside of N.A.
  • 23.
    Brand safety shouldbe of the utmost importance to advertisers. Even one poorly delivered ad can leave a very bad impression Percent of Campaigns with Impressions Delivered Next to Content Deemed “Not Brand Safe”
  • 24.
    The Above-the-Fold Myth? Above-the-foldin-view rates ranged from 48% to 100% source: comScore vCE charter study
  • 25.
    Some Below-the-Fold ads areactually premium inventory Below-the-fold in-view ranged from 3% to 67%. source: comScore vCE charter study
  • 26.
    There is goodnews for advertisers and publishers  Analogous to TV audience guarantees  Eliminating unseen online inventory supply results in:  More effective / efficient campaigns and less waste for advertisers  More accurate metrics for market mix models  Better proof of digital ad effectiveness  Increased transparency/accountability means increased confidence in digital
  • 27.
    Thank You! Download thewhitepaper, continue the conversation

Editor's Notes

  • #12 To get some market reaction, we introduced a charter program of some really large advertisers.We were delighted by the response to the program, and let me take the opportunity to say to them, Thanks you for participatingNow let us talk about what we learned.
  • #15 Prior to the introduction of vCE, there hasn’t been a proven standard technology in the marketplace that could measure all three types of iframes to get a complete read of visibilityMeasuring the first two – directly on the site iframes and friendly iframes is relatively easy, and there have been solutions for these. However, unfriendly iframes, or cross domain iframes, are hard to measure in terms of location on the screen. Unfortunately, a large percent of ads are delivered this way, meaning that missing this piece of the equation, means not covering the entire story. vCE accounts for ALL THREE providing insight into total visibility.
  • #19 Difference of in-view rates between Top 50 sites and long tail sites in their category was a full 16-percentage points in the USEuropean Results:Top 50                  66%Top 100                63%Top 500                61%501+                    55% The only minor difference with the US is that we’ve taken the full range of sites rather than just a category We had half the publisher volume we saw in the US, and a lot have very small numbers of impressions which, when visible, skew upwards the long tail visibility average.  The overall data set tells the best story and is most consistent with the original US slide.
  • #25 To clarify, being in-view has nothing to do with being above or below the fold. In fact, there are some myths the charter study helped to debunk. For some campaigns less than half of the above the fold placements where actually seen.
  • #26 On the other hand some placements below the fold where in almost 70% of the time.