All Possible Worlds  iab mx_public
Upcoming SlideShare
Loading in...5
×
 

All Possible Worlds iab mx_public

on

  • 824 views

Todas las posibilidades en los avances de la medición digital por comScore

Todas las posibilidades en los avances de la medición digital por comScore

Statistics

Views

Total Views
824
Views on SlideShare
824
Embed Views
0

Actions

Likes
0
Downloads
15
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

All Possible Worlds  iab mx_public All Possible Worlds iab mx_public Presentation Transcript

  • All Possible Worlds:Advances in Digital MeasurementP.A Pellegrini, Ph.D. VP International ResearchcomScore, Inc.
  • Agenda• Strategy for Digital Audience Measurement• Current Methodology and Enhancements• Impact of Mobile Devices and Tablets • Fragmentation of Platforms and Consumption • Towards a Digital Measurement Ecosystem • User Panels, Site-Centric Tag Measurement, Network-Centric Logs, Dynamic Empanelment• Establishing Next Generation Best Practices • Probabilistic Reference Point and Validation © comScore, Inc. Proprietary. 2
  • comScore Digital Measurement Strategy © comScore, Inc. Proprietary. 3
  • The Challenge: Devices to be tracked © comScore, Inc. Proprietary. 4
  • The Challenge: Formats to be tracked © comScore, Inc. Proprietary. 5
  • The Challenge: Data sources to integrate © comScore, Inc. Proprietary. 6
  • The Challenge: Processes to be linked © comScore, Inc. Proprietary. 7
  • The scope of the challenge for Digital Media Measurement © comScore, Inc. Proprietary. 8
  • Digital Requirements REQUIRE Depth & Representivity – Endless scope of possible behaviors – Engagement profiles dominated by heaviest users – Must capture micro audiences REQUIRE Integration within broad Media Research environment – Media ecosystem is bigger than just Digital – Methods & Practices must synch with those used in other media REQUIRE full view into the digital consumer across all platforms – Web-only view no longer represents the digital consumer REQUIRE ability to integrate methodologies – Methods that work on one platform may fail on another – Must integrate ‘best-of-breed’ methodologies for complete measurement © comScore, Inc. Proprietary. 9
  • Key UDM Lessons Panel Requirements Staggeringly high – Key segments difficult to represent: Work, Tech Professionals – Systemic under-measurement of certain behaviors: news, developer networks – .. efforts to date to build a “perfect panel” have fallen short Publishers willing to provide Site-centric measurement – Better measurement is valued. Outweighs cost of tagging burden. – Ability to ‘close-the-loop’ to reconcile sources reduces transaction costs  Some Ecosystem elements best measured by tag-based collection – Ad networks, Video & Mobile – Tags allow for rich event attribution (Ad Net reach, YouTube, Hulu) – Coverage across all platforms UDM must evolve in response to feedback – Simple assumptions must yield to more robust methods (cookie/person est.) – UDM highlights key gaps in panel methodology (large deltas, apples/oranges) – Must integrate Site-centric measurements into panel methodology © comScore, Inc. Proprietary. 10
  • Digital Measurement Outlook Drive next-generation Panel Methodology – Leverage Site-census measurements Integrate Digital Measurement into broader media ecosystem – Adopt industry standards: enumeration, demography, sampling – Deliver transparency through public audit reporting – Integrate digital into Agency/Planning infrastructure & 3rd party tools Move Digital beyond the Web – Mobile matters.  So do Tablets… whether we can build panels or not – It is our responsibility to deliver measurement Develop Cross-Platform integration techniques – Anchor methods to site-centric observations across platforms – Accommodate platform-specific methods to deliver a Unified view © comScore, Inc. Proprietary. 11
  • Current Methodology and Enhancements © comScore, Inc. Proprietary. 12
  • Building Blocks of Unified Digital Measurement Establishment Surveys Population Targets EnumerationUnified Digital Measurement Online Recruitment UVs and Page Views Panel Unification Recruitment comScore Unified Digital Measurement Passive Meter Data Collection Traffic Data Session Assignment Dictionary Site Hierarchy Allocation Collection Site Centric tagging Panel Bias Correction Weighting, Preventing Bias, © comScore, Inc. Proprietary. 13
  • Enumeration Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 14
  • Defining the Universe: The Establishment Survey  Accurate estimates of the size and composition of the target populations are critical to audience measurement.  High quality establishment survey enables comScore to accurately estimate the target population sizes for in-market age and gender demographics, penetration, and frequency of use in each market.  comScore currently uses proprietary surveys, high quality government sources (eg, Eurostat), or the local ‘gold standard’ survey.  For expanded demographics and to avoid mobile-only issues, comScore has agreed to partner or license gold standard surveys like the NRS in the UK, PMB Print Measurement Bureau in Canada, The Readership Works by Ipsos in AU, and The EGM survey in Spain. comScore currently measures & reports persons age 6+ who accessed the Internet from either a home or a work computer in the past 30 days. © comScore, Inc. Proprietary. 15
  • Panel Recruitment Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 16
  • Recruitment in the Digital Age Given the issues of: – Scalability (the need for panel sizes in the hundreds of thousands to address the long tail of digital media and fragmentation) – Declining efficacy of RDD recruiting comScore recruits panelists online, in two ways: – The Affiliate Program (“banners-plus-Permission Research”) – Third Party Application Providers (TAP) Pioneers of large, non-probability panels – Panel Dashboard, Jackknife Replication © comScore, Inc. Proprietary. 17
  • Online Recruitment  from Two Approaches… Affiliate Program Third-Party Application Provider (TAP) Affiliate network comprised of web  comScore partners with application entities which meet comScores providers who offer visitors a “quid  quality criteria pro quo” On these sites, panelists are  The web user is offered something recruited via banner ads free (software, applications, utilities etc.) in exchange for exposure to Appeals are targeted through a our panel solicitation broad array of smaller web entities  The user can generally get the free Respondents are directed to our app without joining the panel online intake entity www.permissionresearch.com  New “Value Proposition” (in Affiliate  and TAP): Trees for Knowledge © comScore, Inc. Proprietary. 18
  • Permission Research Panel Recruitment – Creative Examples © comScore, Inc. Proprietary. 19
  • TAP Recruiting Experience User is on the Internet searching for software applications, or some other download This user is looking for “photo animation  software” © comScore, Inc. Proprietary. 20
  • TAP User gets desired search result and clicks through to site © comScore, Inc. Proprietary. 21
  • TAP – Partner Site Once on the site, user decides to initiate the free download © comScore, Inc. Proprietary. 22
  • TAP – Installation Flow Begin Installation process © comScore, Inc. Proprietary. 23
  • TAP – Installation Flow Accept the License Agreement of the value proposition software (this is the partner’s  agreement) © comScore, Inc. Proprietary. 24
  • TAP – User Acceptance User is also asked to join the comScore panel. This is a separate step from the partner’s  software acceptance User must take a positive action (not pre-selected) © comScore, Inc. Proprietary. 25
  • TAP – Partner-side Demo Collection Take a short survey where we collect basic demos © comScore, Inc. Proprietary. 26
  • Value Propositions from TAP RecruitmentA sampling of the many value propositions offered….PC Utilities and Productivity Desktop Personalization and Digital Media Applications Games and Entertainment Tools Appearance•Alarm Clock •Screensavers •Audio/Video Converters •Arcade Games•Application Launcher •Icons •Audio Editor •Board Games•Checklist Software •Images •Audio Extractor •Fantasy Games•Download Accelerator •Smileys •CD and DVD Burners & •Global Radio & TV Stations•Download Manager •Emoticons Rippers •Lyric finder•Media Streamer •Video Converter for •Media Players •Music Downloads•PC Access Control Screensavers •Media Downloaders •Photo Album Software •Icon File converter •Media Search Tools •Photo Morphing Software•PC Customizable Shutdown •Wallpapers / Themes •Media Library/Organizer •Quest Games•PC DVR •Screen Pen •Ringtones•TV Optimization Software •Sports Games•Windows Application/PC •Strategy Games Lock •Online Games•File Protection •War Games•PC Power Saver•Online Disk Storage•History Cleaner•Internet Usage and Performance Stats•Unit Converter/Metric Converter•Currency Calculator/Converter•Customizable Binary Converter © comScore, Inc. Proprietary. 27
  • Trees for Knowledge In 2008 comScore launched a new recruitment appeal, for both Permission Research and TAP panelists – Under our Trees for Knowledge program, comScore has partnered with Trees for the Future to plant a tree for new members that join the panel – comScore’s initial donation will support the planting of one million trees in  developing communities throughout the world Creative made available to affiliates to promote campaign Permission Research offer: plant a tree at installation; then one tree for each month the panelist stays in panel © comScore, Inc. Proprietary. 28 28
  • Passive Data Collection Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 29
  • comScore’s Privacy Protected Proprietary Collection Software with Explicit Panelist Permission comScore’s cProxy meter allows us to “see” user activity at the device or screen  side (user experience as opposed to site-centric) without using cookies■ comScore captures: – URL – Engagement (active versus passive) – Keystroke and mouse activity and intensity – Information passing between the user and the Internet entity – Ads delivered, whether clicked or not – Application installation and usage (Excel, Word etc.)■ Data capture tiers – HTTP: comScore can capture just about anything over the HTTP protocol (including HTTP and HTTPS) – Proprietary technology for real time measurement of advanced protocols like streaming – Measurement of AOL proprietary and IM engagement © comScore, Inc. Proprietary. 30
  • Session Assignment Technology (SAT) © comScore, Inc. Proprietary. 31
  • Who is Using the Device? In the digital age, tracking the behavior of devices is easier than ever before. Moving from devices (which don’t buy products or view  advertisements) to people still poses unique challenges. In panel measurement, the ideal is passive observation over an extended time period. One approach: use “who are you?” pop-ups. – Requiring respondents to self-identify each user session (similar to traditional media construct used in TV People Meter) – Intrusive – Fatiguing – Biasing Intrusiveness drives panel turnover to unmanageable levels. © comScore, Inc. Proprietary. 32
  • Who is using the device? – Passive Biometric Observation  comScore develops a unique biometric signature for each panelist. – Session Assignment Technology (SAT): a proprietary, patented technology that creates biometric “fingerprints” for each person using the device. Family Roster: Mouse Activity: Email, Age, Sex… Double Click Speed, Unique Profile of Each Mouse Movement User in a Household Sharing a Computer Online Accounts: Keyboard Activity: Cadence tracking of 100+ keywords © comScore, Inc. Proprietary. 33
  • Bias Correction Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 34
  • Preventing Panel Recruitment Bias At the recruitment stage, this is handled by 3 strategies: – Hundreds of diverse incentives – Over 40 different partners, no access panelists or ‘take a survey’ partners – Multiple advertising links to partners provide huge reach over each campaign Provides extensive and broad reach across online universe to minimize potential recruitment bias. Additionally, both demographic AND behavioural weighting (based on census information); the latter removes recruitment bias. Ultimately, the controls in UDM are census based rather than sample based, and therefore they overcome any recruitment bias and sampling error of either a calibration panel or probability sample. © comScore, Inc. Proprietary. 35
  • The Recruitment Dashboard: Online Recruitment Balancing Recruitment targeting allows more control over marginal recruitment via development of partner dashboard. The dashboard uses reports to track partner yield by demographic group, over time Current Month Index Last Month Index 2 Months Ago Index Partner1 Target In Tab vs UE In Tab vs UE In Tab vs UE Target In Tab % UEAll_2-5 2.3 2.8 2.7 All_2-5 50 0.8% 2.0%All_6-11 4.4 5.3 4.9 All_6-11 50 0.8% 8.1% All_12-14 0 0.0% 5.7%All_12-14 15.1 43.9 42.7 All_15-17 0 0.0% 6.0%All_15-17 114.7 154.4 151.6 All_18-24 1,600 26.7% 13.9%All_18-24 203.4 193.4 188.0 All_25-34 2,000 33.3% 18.0%All_25-34 122.4 118.0 117.7 All_35-44 1,800 30.0% 18.1%All_35-44 91.1 92.0 91.6 All_45-49 500 8.3% 8.8% All_50-54 0 0.0% 6.7%All_45-49 81.5 79.7 80.8 All_55-64 0 0.0% 7.6%All_50-54 89.4 78.5 82.3 All_65+ 0 0.0% 5.0%All_55-64 96.3 82.5 84.4 Total 6,000 100.0% 100.0%All_65+ 89.8 83.0 90.5 © comScore, Inc. Proprietary. 36
  • Traffic Allocation Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 37
  • Web visitation and traffic allocated based on 6 levels of site hierarchy © comScore, Inc. Proprietary. 38
  • Client Focus Dictionary (CFD) – Media Entities Traffic allocated across – 118 content categories and sub-categories – Six levels of hierarchy supporting a “parent/child” relationship with intelligent grouping of properties – Ad Networks providing potential and actual reach – All platforms including internet and mobile Building block for each level in the hierarchy is a URL pattern – Example: %yahoo.com%sports%cricket% Allows calculation of key inputs to Unified Digital Measurement at as granular a level as a URL pattern resulting in – Accurate estimates of crucial metrics like cookie deletion, audience overlap etc – Bias elimination created by grouping different types of sites into one broad bucket © comScore, Inc. Proprietary. 39
  • Unification Enumeration Panel Unification Recruitment comScore Unified Digital Measurement Traffic Data Allocation Collection Bias Correction © comScore, Inc. Proprietary. 40
  • Fundamental Problem of Site Centric Measurement:No Unique User IDWeb Analytics Approximation Unique User = Cookie ID (if Cookies can be set) or IP Address + User AgentMore Problems…• Cookies are deleted, and when they are, the same user can be counted multiple times• IP Addresses change all the time causing inflation of user counts• In any case, a UU is an attempt to identify a machine (or a browser), which can represent multiple people or a fraction of the usage of a single person• Some 56% of all machines had a session on more than one browser, and since each browser sets its own cookie on a machine, they can’t de-dup. © comScore, Inc. Proprietary. 41
  • The Impact of Cookie Deletion:Repeat Visitors are counted Multiple Times The site reads THREE distinct cookies, which means this ONE visitor is counted THREE times. © comScore, Inc. Proprietary. 42
  • Unified Unique Visitors UDM combines cookies at census level with panel insights to report unique visitors for the combined home and work audience. Methodology: – Collect 3rd party cookies – Filter cookie data to include: User generated calls by excluding bot and spider traffic (IAB lists) Country specific traffic by excluding traffic outside reportable market Home, work, mobile and commercial shared use audience – Use panel to understand cookies per person (explained in the next slide) Based on entity-specific user behavior – Some visitors to a entity may not receive a comScore cookie due to non- tagged portion of the entity comScore accounts for these visitors using the panel. – Visitors are then de-duplicated across home and work for both unique visitors with and without cookies. © comScore, Inc. Proprietary. 43
  • Cookies per Person Advertisers demand people measurement, not cookies or browsers Cookies are not people due to duplication at the cookie level caused by several factors including: – Users deleting or blocking cookies – Multiple browsers per device This is what differentiates UDM from other ‘hybrid’ methodologies; – Multiple devices per location consistent with Global Industry – Multiple users on the same device recommendations comScore calculates cookies per person each month at the site level by country using panel observations – This factor is applied to cookies observed for the entity to remove duplication and derive Unique Persons Cookies per person may be seen as a function of: – Site usage (see above) – Visitation frequency – Usage intensity © comScore, Inc. Proprietary. 44
  • Total Universe: Unique Visitor Calculation © comScore, Inc. Proprietary. 45
  • UDM Traffic collection We collect traffic (tag counts and cookies) from various locations, devices and content types that are tagged • Home • Work • Libraries Locations • Universities • Internet Cafes • Other • PC • Mac Devices • Mobile phones • Tablets • Other • PC based web Content • Mobile web Types • Apps • Other © comScore, Inc. Proprietary. 46
  • Total Universe Page Views Segment Traffic by Total Universe Location / Device Page Views Mobile Phones Shared Machines Work Home Filtered Census Traffic © comScore, Inc. Proprietary. 47
  • Traffic Allocation Segment Traffic by Total Universe Location / Device Unique Visitors Home Work Home + Work Mobile Other Multiple Shared Filtered Web Apps Schools use machines Census Traffic Libraries & Other © comScore, Inc. Proprietary. 48
  • Impact of Mobile Devices and Tablets © comScore, Inc. Proprietary. 49
  • Device Fragmentation beyond the PC is starting todefine Digital Media © comScore, Inc. Proprietary. 50
  • Device Fragmentation Connected Devices per HH 60% % of Households 50% 40% PC Devices 30% All Devices 20% 10% 0% 1 2 3 4+ # of Devices in HH Mobile Devices & Tablets are Driving trend in Multi-device Access © comScore, Inc. Proprietary. 51
  • Device FragmentationMore than 50% Connected Devices per HH of Internet 60%Households still have a Single % of Households 50% PC 40% PC Devices 30% .. But Nearly All Devices 40% of 20% Internet Households 10% have more than 3 0% Connected 1 2 3 4+ Devices # of Devices in HH Mobile Devices & Tablets are Driving trend in Multi-device Access © comScore, Inc. Proprietary. 52
  • Behaviors are different across devices.PC behaviors do not represent activities across alldevices © comScore, Inc. Proprietary. 53
  • iPhone & iPad User Incremental Usage vs. PC iPhone & iPad Users: The Platform is Incremental Reach over PC the Message: 60.0% 56.8% 50.0% 40.0% Behaviours vary 28.9% 30.0% 26.8% 23.6% 21.7% by platform 20.0% 12.5% 8.7% 10.0% 6.9% 0.2% 0.0% Information e-mail Maps Business/ Networking Total Internet Retail Sports Navigation Finance Search/ News/ Social Incremental 2.0x 9.2x 1.6x 1.2x 1.9x 1.3x 2.8x 2.5x 1.4x Duration Source: comScore custom research; experimental iPhone/PC Overlap panel © comScore, Inc. Proprietary. 54
  • Trend toward a mutli-platform digital world is justbeginning © comScore, Inc. Proprietary. 55
  • Platform Explosion is just getting started 40% US Smartphone Penetration: 33.5% 35% 30% Smartphone Population 25% Growing Strong 20% 50%+ Annual Growth 15% 10% US iPad Penetration: 4.4% 5% 0% Nov-2010 Feb-2011 Feb-2010 Mar-2010 Mar-2011 May-2010 Jul-2010 Aug-2010 Sep-2010 Dec-2010 Apr-2010 Oct-2010 Apr-2011 Jan-2010 Jun-2010 Jan-2011 Jun-2011 May-2011 Source: comScore Mobilens US June 2011 © comScore, Inc. Proprietary. 56
  • Platform Explosion is just getting started 100% 90% 66% of 80% Americans 70% DO NOT yet 60% have a Smartphones 50% 40% US Smartphone Penetration: 33.5% 30% 20% US iPad Penetration: 4.4% 95% of 10% Americans 0% Nov-2010 DO NOT yet Feb-2011 Feb-2010 Mar-2010 Mar-2011 May-2010 Jul-2010 Aug-2010 Sep-2010 Dec-2010 Apr-2010 Oct-2010 Apr-2011 Jan-2010 Jun-2010 Jan-2011 Jun-2011 May-2011 have an iPad Source: comScore Mobilens US June 2011 © comScore, Inc. Proprietary. 57
  • Unified Digital Measurement™ (UDM)   Global PERSON Global DEVICE Measurement Measurement PANEL CENSUS Unified Digital Measurement (UDM) Patent-Pending Methodology Adopted by 80% of Top 100 US Media Properties & 60% in U.K. © comScore, Inc. Proprietary. 58 V0910
  • Digital Measurement Ecosystem Network Logs User Site Panels Tags Household Application Panels Tags © comScore, Inc. Proprietary. 59
  • Technical Measurement Review Platform Network Publisher Format Demo Coverage Coverage Coverage Coverage AvailabilityMetered Panel Poor Very Good Moderate Moderate Very GoodPublisher Tags Very Good Very Good Poor Moderate PoorNetwork Census Good Poor Very Good Good Poor © comScore, Inc. Proprietary. 60
  • Technical Measurement Review Platform Network Publisher Format Demo Coverage Coverage Coverage Coverage AvailabilityMetered Panel Poor Very Good Moderate Moderate Very GoodPublisher Tags Very Good Very Good Poor Moderate PoorNetwork Census Good Poor Very Good Good Poor Site Tags provide foundation for full cross platform view. Drives de-duplication methodology © comScore, Inc. Proprietary. 61
  • Technical Measurement Review Platform Network Publisher Format Demo Coverage Coverage Coverage Coverage AvailabilityMetered Panel Poor Very Good Moderate Moderate Very GoodPublisher Tags Very Good Very Good Poor Moderate PoorNetwork Census Good Poor Very Good Good Poor Panels provide foundation for person-centric measurement & demographics © comScore, Inc. Proprietary. 62
  • Technical Measurement Review Platform Network Publisher Format Demo Coverage Coverage Coverage Coverage AvailabilityMetered Panel Poor Very Good Moderate Moderate Very GoodSite Tags Very Good Very Good Poor Moderate PoorNetwork Census Good Poor Very Good Good Poor Publisher coverage remains a challenge. Site tags will remain most accurate measurement © comScore, Inc. Proprietary. 63
  • Establishing Next Generation Best Practices © comScore, Inc. Proprietary. 64
  • A Next Generation Digital Measurement System Industry Oversight Define Universe & Control & Audits by Independent for Demographic Bias Agents High Quality Enumeration Survey Active Panel Census Panel Calibration Partner ControlControl Dashboard PanelBehavioral In-tab Monthly CompositionBias Sample Probability Enhanced Validation Sample Panel Selection Probability Jackknife Validation Replication Panel Study Panel Stability & Calculate Margin of Error © comScore, Inc. Proprietary. 65
  • Probability Validation Panel: Concept & Explanation • Non-probability sampling in digital user panels • Sampling challenges • Statistical efficiency and effective sample size • Jackknife replicates, probability and validation • Empirical method to examine variance in sample data • Provides standard error, design effect, stability over sample sources • Assumes EDF is good estimator of PDF, very large samples,… • PVP triangulates on bias and reliability questions • Census as a source for behavioral weights • Panelist cookie consumption versus universe cookie consumption © comScore, Inc. Proprietary. 66
  • Standard Error Study using Jackknife replication Jackknife replication, related to the more general Bootstrap, is used to assess the variability of a statistic by examining the variation within the sample data (vs parametric assumptions), especially with complex sampling (Efron & Tibshirani, 1994). Variance of standard comScore key measures estimators (for monthly, annual, including top 100 properties & associated entities): – Unique Visitors – Page Views – Duration(Minutes) Purpose is to understand the variance of the sample design, controlling for comScore’s methodology and processes for any monthly delivery. Results of the study will be utilized to create a general functional form for estimating margin of error for any reportable data. © comScore, Inc. Proprietary. 67
  • Unique Visitor replication results: Smoothed polynomial showingconsistency of UV estimates month over month © comScore, Inc. Proprietary. 68
  • Page View replication results: Smoothed polynomial showingconsistency of PV estimates month over month © comScore, Inc. Proprietary. 69
  • Jackknife Validation continued… Replication study shows consistency of estimates across properties of various sizes and categories, and across time periods. Provides empirical data to develop a polynomial function for estimating margin of error for any reportable data. Addresses key concern/question about variance estimates from non- probability samples and their stability over time, given the fluctuation of sample sources (partners, affiliate sites). Also powerful to validate bias correction using device census calibration – there are important assumptions, however: – But Jackknife assumes EDF is representative of the PDF. While this is mitigated by large sample sizes, it requires validation…hence, PVP. © comScore, Inc. Proprietary. 70
  • Summary • Digital Measurement Challenges are unprecedented • Fragmentation of devices, consumption, attention • Multi-source and mixed methods are required • Measurement across platforms and locations • Site tagging, OS and IP information plus user panels • Duplication issue across multiple access points • Differences in patterns of duplication across content types, and demos • Industry standards must be set; industry participation is crucial • Probabilistic reference for multi-source measurement • Guide to industry accepted methodology and best practices © comScore, Inc. Proprietary. 71