Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

My concern about media audience measurements tamara silina - feb 17th 2013


Published on

  • Be the first to comment

  • Be the first to like this

My concern about media audience measurements tamara silina - feb 17th 2013

  1. 1. My concern about media audience measurments Tamara Silina, February 2013
  2. 2. A few months ago, I had a quite unclear understanding of what audiencemeasurement is. After been involved in managing marketing for a new local radiostation, I first looked at how decisions were made by sellers and buyers in theadvertising space market. I was rather disappointed by what I found.To summarize, I would say there are so many errors, bias and dysfunctions intraditional medias audience measurement that it seems it is very risky to takedecisions based on this information.Thank you to professionals in this field who answered my questions, who shared theirknowledge and opinions with me through my blog and who read back my posts indespite of my ingenuous ignorance.And sorry for my poor English. TamaraThanks to Orson Welles for the Citizen Kane few images I borrowed.
  3. 3. It is not aboutcreating panic, but to first question: are we inform… all going to die in horrible suffering? Patrick Chapatte, 28 février 2006, Le Temps.
  4. 4. What I heard from mediaWhat I have heard about media strategiesWhat purpose do audience measurements serve?New technical solutionsWhat we shall have to face
  5. 5. Here you are 5
  6. 6. What I have heard about old traditional media strategiesI selected three key ideas about what I was told about old media strategicanalysis of old traditional media groups. The share of the traditional media (radio, TV, print...) in the ad space market should decrease because of the inevitable decline of their audience. Ad expenses should be transferred mainly to digital media and other new areas of marketing investment. Their strategy would be to extend as much as possible the period during which traditional media can still be profitable.To do this, it would be vital for them not to disturb the factors that keeps theirbusiness especially audience measurement systems. 6
  7. 7. What I have heard about new and small traditional mediaNew entrepreneurs in traditional media business make different analysis: Today, It is much easier and less expensive to create and operate new traditional media with new production and broadcasting technologies and new content sources. New and small media would be more in synergy with Internet social networks. Their potential development would be higher because of their ability to deal with proximity, community, and generation issues through targeted and niches strategies.But, as far as their audience is not measured by the current audiencemeasurement systems, their development is strongly limited. 7
  8. 8. But what is the use of audience measurements? 8
  9. 9. What I understood about the purpose of audience measurmentAudience measurement is mainly used in the advertising space market ,to givea value to , and promote ad space. It also allows content managers to identifywho is listening, watching, reading, seeing what, in order to adjusts theircontents. But it is mainly the first function that determines the configuration ofaudience measurement.For advertisers and advertising agencies, audience measurement allows,before a campaign, to identify spaces with the best cost / potential impact ontheir target. During and after the campaign, it enables to evaluate somecampaigns performances. 9
  10. 10. In few words, how are audience measurement made?Who take decisions? Associations of media, agencies and advertisers decidewhich methods and means to implement. They choose, after bidding, researchagencies.How is it done? For radio and print, telephone or self-administered surveys areconducted. They may be continuous or by waves. Television audiencemeasurement use diaries surveys or people meters panels. For outdoor, they usesurveys or GPS tracking.Some few systems use automatic individual people meters (radio and TV).Who pays? These surveys and panels are mainly funded by the media whowant their audience to be measured. 10
  11. 11. Lie to me 11
  12. 12. Problems?In trying to understand these processes, I was very surprised by malfunctionsthat I did not expect to find in one of the first markets in the world economy.I discovered that most transactions are based on approximate data andtechniques that would have horrified brokers and traders in the fifties. 12
  13. 13. 17 key problems 1. Systems do not match the needs of data users, 2. Measures that are not cross - media, 3. Poor definition of the population whose audience must be measured, 4. Weak segmentation and profiling, 5. Irrelevant size and structure of samples, 6. Wrong survey techniques, 7. Bias in recruiting, 8. Question that induce untrue answers, 9. Irrelevant delay and frequency in data collection and delivery,10. Irrelevant presentation of delivered data,11. Delayed access to the data that make it obsolete,12. Non-compliance with specifications,13. Lax controls,14. Sidelining of the real experts,15. Dishonest data manipulation,16. Resistance of main media to change,17. Resistance of media research institutes to change. 13
  14. 14. 1. Audience measurements don’t match the needs of data usersThe first audience measurement function should be to help advertising spacebuyers and sellers, to setup space "value".To optimize and fluidify transactions in a market, the required information fordecision makers should be available by streaming and in real-time, throughconsistent tools for consultation, analysis, placement of order and post-evaluation.In all structured markets, these functions are integrated into homogeneous,optimized and controlled systems. In the advertising space market, thedata about programs and commercials monitoring, audience and forecasts,and the tools to exploit them are provided by heterogeneous suppliers withoutglobal vision of the users real needs, with virtually no control. Each of thesesuppliers thinks he is doing its best just by coordinating with others. This type oforganization produces a low result. 14
  15. 15. What current systems measure, is mainly the exposure of respondents orobserved members of panels, to what media broadcast. But is this concept ofexposure still enough to define the value of the advertising space?Don’t you think that, for example, involvement, interest, attention,appreciation, loyalty to programs, would be taken into account by theadvertisers and agencies if that data was available?Couldn’t we take into account the relationship between the consumers andthe products?The audience measurements are designed today without a global visionand without taking into account the real needs of those who determine theeffectiveness of advertising space and who develop the market space. 15
  16. 16. 2. No cross media audience measurementFor a potential buyer, the value of an offer advertising space, is related to histarget audience compared to those of all the alternative media and to theircomplementarily, for the target coverage.Media planners work is not only finding spaces that cost less by contact, but alsoidentifying space assemblages of complementary media to optimize therepetition on a specific share of the target. This is possible by spreading messagesthrough different types of media. Decision makers need to evaluate at the sametime the cost and complementarily of ad spaces of TV channels, radio stations,newspapers, etc.. This requirement has been explicitly expressed by internationalassociations of advertisers and, in 2009, by The Coalition for Innovative MediaMeasurement (, each year, new panels and new audience researches are setup without they taking into account this requirement. It seems that associationsand groups who control audience measurements do not understand the issue. 16
  17. 17. 3. Inadequate definition of what is to be measuredMore and more media are cross- national. For a radio or a television stationwhich broadcasts by satellite or on the Internet, the geographic area of itspotential audience is a linguistic territory. For example, Russian-speakingchannels are viewed in fourteen countries and Francophone channels, in fortycountries.Many advertisers who work in several countries have already internationalizedtheir branding positioning according to linguistic and cultural territory. Their adspace purchasing decisions are based on audience in these areas. To do this,some of them have to conduct their own multi-country measurements.International media as those who provide both satellite and cable ,have accessto inconsistent and heterogeneous audience measurement data. This is whytheir advertising space is under-estimated. 17
  18. 18. 4. Irrelevant segmentation and profiling toolsThe media audience is fragmented, it is more individual and independent. Newframeworks are needed to understand behavior and segmentation. Despite this,people who are recruited to participate in surveys and panels are brieflydescribed and profiled. In fact, in most cases, the minimum criteria is used,especially for television panels.Yet the media, advertisers and agencies communicate their targeting criteriaand segmentation. They need segmented data according to these criteria fortheir decisions. ( Page 5).Because they do not have such data, media planners are used to make riskyprojections and interpolating.A paradox: despite the fact that respondents are defined through only basiccriteria, it is generally very difficult and expensive to obtain data extractions usingthese criteria.What could drive those who designed these systems? Did they know thatrestricting samples qualification and profiling and reducing access to nativedata, would directly affect the media space demand? 18
  19. 19. 5. Irrelevant samples size and structureAll parts of a population are not worth the same as advertising targets. Wecould therefore expect that the targets who generate most ad expenses,would be covered by more accurate and more frequent measurements.But sample surveys and panels are most of the time structured as reducedmodels of population, whose audience has to be measured while they shouldover-represent segments of the population that are targeted by mostcampaigns.For this reason, a large part of the data is nearly never used.With this model logic, to get large enough useful sub-samples, it is necessarythat the size of the entire sample is very large, which ultimately would result inadditional - and hardly justifiable- costs.How is it possible that those who designed these systems, did not askthemselves what information was really useful? Why were they so anxious tohave very large samples, so unnecessarily representative of the population thatis useless to data users? 19
  20. 20. 6. Survey system which does’t match all segmentsThose who decide how audience measurement should be always want to use amonolithic methodology to cover the entire population, all segments, all profiles,the same way. This is probably in order to be able to compare audiencebehavior of different segments. These doesnt match any need of any data user.It is common sense that no data collection method can match all profiles. Someare more adequate to some segments and others to other segments.Those which can match for average profiles are not effective for higher socialcategories. For instance, television people meters are not able to measure theaudience of high purchasing power consumers, who, by the way, are subject toheavy advertising investments… 20
  21. 21. 7. Recruitment methods biasesI was lucky to be in touch with some media audience research teams. One ofthe things that disturbs them the most, is the very low representativeness ofpeople who are recruited for these measurements.One key factor is the low level of acceptance to participate in this kind ofsurvey or panel. A large part of the samples seems to be very difficult torecruit. For phone surveys, 90% refuse and 99% do for panels & people meters.It is very, very far below standards to get representative samples.Some profiles - especially in the higher social categories – are just impossible torecruit. This means that some key targets are just not represented in thesamples, even if it was in the scope of work and even if they appear in thesample description of the research institute.Contrary to what is sometimes said, no weighing method can correct theseproblems.Unfortunately, samples are widely made of atypical people who cannotrepresent real audience 21
  22. 22. 8. Questionnaires that produce wrong answersFor radio and print, most audience measurements are using self-administeredquestionnaires or are administered by phone or Internet.For radio, several researches have shown that a large part of peopleanswering, often don’t know what radio station they are listening to. And alarger part of them don’t remember what stations they had listened to the daybefore . At the same time, the most used information collected by thesesurveys is the last day audience.It is also shown that respondents tend to say the names of stations with a strongawareness or reputation, rather than to say that they do not remember. Thisgreatly benefit the most well known radio stations. Some of the respondentsalso say the names of the radio stations for which they have sympathy. Similarphenomena are observed for newspapers and magazines audience.,%20diversity,%20and%201a.pdfThe result is that these survey methods induce wrong answers and more often,media audiences which are better known ,are overvalued at the expense ofothers. 22
  23. 23. 9. Dysfunctions due to delays and frequencyRadio and print audience measurement surveys are conducted four or fivetimes a year in most countries. They need four or six week to be conducted anddata are provided with a delay of several weeks. However, most campaigns lastno more than three weeks.How could - we justify the quality of the space we sell with such delays? Imaginea buyer who will receive information on the effectiveness of his campaignsseveral months after? Some advertisers have to conduct their own audiencesurvey to be able to adjust their campaigns.It is sometimes said by some experts about radio and press audience thataudience vary little. But it is not true, as shown by researches using daily surveys(Arbitron, Radio audience measurement in New York using PPM). 23
  24. 24. 10. Irrelevant data access and deliveryData supplied by research institutes that produce audience measurementscannot be accessed through query tools that would cross numerous criteria.However, such software are available.According to some professionals, it appears that direct access to data wouldmake it easy to detect inconsistencies, oversized weighing and too lowrepresentation of some profiles in the samples. 24
  25. 25. 11. Information are too old when deliveredIn an ordinary market place, the more and the faster information is available,the stronger is the market active. In most markets, everything is done to get realtime data. Thats what I thought until I worked for a radio station.In most radio stations, like the one I worked for, audience measurements aremade using a telephone survey. About a thousand people are interviewed oneverything they heard the day before. One might say that data every week, isnot so bad and it could stimulate the market. But it is not the case. The data isprocessed once per quarter and it takes almost six weeks to have access to theearliest data.So by the end of January, I have data from November. I cannot use the datato justify my new year prices. Advertisers cannot make a decision and they donot buy my space. Why not treat and deliver data in a timely fit? 25
  26. 26. 12. Non-compliance with tenders specificationsThe impossibility to reach norms, poorly controlled recruitmentspecifications and data adjustments, drive institutes to use solutions thatdegrade the quality.Some profiles, for which recruitment is too difficult or too expensive, arereplaced by people out of the quotas: people of higher socio economic classare replaced by lower class; people in deep rural areas are replaced by nearcity countryside people, or suburban individuals, etc…Some abusive weighing complements abusive recruitment. Subsamples ofdifficult to recruit profiles, are over-weighed well beyond what can bestatistically acceptable. Some people may be counted ten times.These practices are the main reason for the barriers that are erected to accessindividual data from surveys and sorting purposes. They would make themvisible.We should be wary of audience measurement systems which do not giveaccess to individual data to any user, even for reason of consistency check. 26
  27. 27. 13. Lax controlsIn modern markets places such as stock exchange, controlling systems are verystringent. They focus on the design of systems, their operation, their integrity andthey check absence of abuse in the use of data. Each of these controls isassigned to a separate institution and whose activity in only controlling andauditing.For audience measurements, the practices are much less stringent. These controlsare organized through committees in which members participated in the choiceof the institute or are involved in audience measurement operations.The assessments are rather assigned to individual experts whose methods maylack rigor and whose independence is not validated. Often, these experts haveworked in audience measurement leading institutes. In addition, these controlsare very infrequent.Sometimes experts deliver two versions of their report: one for wide distribution,with very little criticism, and another for the committee, which it is much morecritical. In these cases, this shows who is the final client of audiencemeasurements. 27
  28. 28. 14. Real experts sideliningExperts who are requested by associations for the design and control ofaudience measurement systems have, most of the time, been involved inoperating this kind of survey or panel. They are not scientists, they aretechnicians. They just reproduce what they know, that is to say, the oldsystems, and to endorse the currents practices.However, there are researchers who work and publish on these topics. Ifound many papers that addressed the problems I have identified. Theseresearchers are never asked for advising or technical auditing.Researchers that I could speak with are pessimistic about change inaudience measurement systems because they think the currentorganizations that drive them doesnt match the new required systems andthey will not ben revolutionary as they should be today. 28
  29. 29. 15. Intentional alteration of dataMore serious than the dysfunction which are induced by aberrant specifications,in some places, data are sometimes intentionally altered, that is to saydeliberately changed. This happened at the request of media, contentproducers or agencies.Owners of media conglomerates explained me how they had to pay to be sureto get good audience.Some engineers that programmed software for media audience measurementexplained to me how audience of some channels can be routinely boosted bya discrete weighting.Moreover, some media planners have detected daily identical ratios for sometime slots per between the main channels, and for long periods.All these examples are recent.These practices may exist because the controls are really insufficient. 29
  30. 30. 16. Leading medias resistance to changeGroups and committees that lead measures audiences measurements areoften dominated by mainstream media. They manage to impose themselvesbecause they have an historical leadership and because they pay more. It istherefore difficult to oppose their views.The orientations of these mainstream media are naturally more conservativebecause they believe the relation between share of spend (ad expenditure)and share of voice (share of audience) and current audience measurementsystems protect their share of voice dominance.The subscription amount for a media to be measured is often used by thesemainstream media as a barrier exclude small media groups.It is clear that if audience measurement were more cheaper, the leadership ofthe mainstream media would be weakened. That’s why they are more fond ofheavy methodologies (with large sample) and they denigrate innovations thatcould reduce cost. 30
  31. 31. 17. Leadings institute resistance to changeAudience measurements are highly profitable and not risky for leading researchagencies.. In this time of crisis for the marketing research market, they are seenas a very stable part of the business.A contract for audience measurements generally last several years, it leaves amargin over 30% and probability of renewal is high. Few industries have bothsuch a level and a so low risk.For television audience measurement, there is a technical barrier to entry whichmakes virtually impossible to win tender for a challenger institute.In some cases, major media are participating in the capital of institutes whomeasure their audience. This reinforces their conservatism. In other marketplaces, such a conflict of interest would be prohibited.Sometimes, media research institutes are not afraid of conflicts of interest maysell services to the media. "It is as if a manufacturer of thermometers sold someaspirin." (E. Leser Slate. Http:// institutes will not add pressure to break the status co. They would fightagainst new technologies that could reduce the cost of media research andopen the door to new institutes competitors. 31
  32. 32. New technical solutions
  33. 33. New available systemsThese are the Arbitron PPM, The GfK MediaWatch, Ipsos MediaCell, INVENTIVETechnologies and maybe others I have not yet identified.All of them can be used for both television and radio. They are portable andindividual.Most of them use watermarking technology. They can measure only theaudience of channels which signal include a special code.Most are not integrated into overall systems of information and analysis for thepreparation of decisions of buyers and sellers of ad space. 33
  34. 34. Arbitron PPMThis Portable People Meter is a small box like a pager. It use watermarking. Someversions could provide real time. Versions currently used transfer their data once aday.The fact that it is a case to carry always with you may be a handicap. "Its a bit likethe electronic bracelet of custody" as described by one panelist.This is a closed system (hard and software), highly protected by patents. Thus, itcan evolve very slowly, more slowly than technologies on which it relies, includingtelecom, Internet and recognition signals. For this reason, it gradually loses itsadvance.It has been many concession contracts and partnerships with major institutions thathave not used a 34
  35. 35. GfK MediaWatchIt is also a passive individual mobile people meter, integrated into a watch.Its main advantage is that it is constantly kept by the panelist.It does not provided in real time data because it is not connected. The panelistmust daily or weekly connect on a rack to upload data.A module can integrate the newpapers and magazines audiencemeasurement (declarative).It also use watermarking, in spite the fact Gfk-Telecontrol had pushed far enoughresearch in recognition of media signals without watermarking.GfK Telecontrol did not take into account that a watch is a very personal object,which carry the image and the style of the person who holds it. They developedquiet ugly and very recognizable watch. This bias is a strong handicap forMediaWatch panel recrutement.It is also a closed system (hardware and software) and therefore it can evolveonly slowly while the technological environment is changing 35
  36. 36. Ipsos MediaCellThis is an app for recognition of sound signals with watermarking which can beincorporated in smart phones. It transmits in almost real time.This is the device that would be the ideal if the watermarking algorithm had notcapacity as low recognition level in noisy environment (discussion, engine, etc.).This is not a hardware closed system like the two previously mentioned systems. Itwill benefit other phone features such as localization, audience measurementbehavior of mobile Internet. This is one of the systems most likely to 36
  37. 37. INVENTIVE TechnologiesThis system also uses panelists smart phone and tablets. It is the only system whichdoesnt use watermarking.The recognition technology for TV channels and radio stations can distinguishtheir signal from strong ambient noises. It can measure the audience of anychannels and any radio stations without limitation. It measures audience share, itprovides audience data in real time (2-3 minutes). It includes a module formeasuring print audience (declarative).It is integrated into a larger system with advertising monitoring tools and 37
  38. 38. Digression on watermarkingThe use of this technology requires each station / channel that wants to measureits audience incorporates a numeric code in the audio signal.It does not measure audience of unmarked channels and stations. Audienceshare, which is known to be the first indicator used for content audience analysiscannot be measured.It is almost never mentioned that the watermarking signal recognition ismalfunctioning for the radio in noisy environments (such as in cars, then that is oneof the contexts in which the hearing of terrestrial radio is still virtually notchallenged by Internet).The few independent technical tests conducted on traditional television peoplemeters (which use watermarking) have an average error of 10%. Tests even rareron passive people meters for radio rated more than 40% error. (Joe way marketing research institutes do forcing on watermarking is surprisingwhen you know its low performances. The reason of this preference could be dueto the fact it allows to force the media to pay to be measured.The individual passive audience measurement can be performed withoutwatermarking and improved performance. 38
  39. 39. What to do for my radio? 39
  40. 40. What theory says?”Decision makers prefer alternatives with less uncertainty. Information reducesuncertainty. So buyers and sellers prefer bids supported by the latest, the mostrelevant, the most exhaustive, the most reliable, the most continuousinformation and in larger quantities”C. Shannon (1948) « A Mathematical Theory of Decisions and Communications ».That’s why advertising space buyers divest traditional media because theoffer is not supported by this relevant information. Audience measurementsand buying and selling space systems them are handicaps for thedevelopment of this market. Well have to change that. 40
  41. 41. What is said by the most advanced professionnals?The required system to replace those currently operating is clearly outlined bythe Coalition for Innovative Media Measurement and by experts.It should measured audience behavior by the same source (single source) andin real-time, by any channel, for television, radio, Internet, print and outdoor. Itshould include real time advertising monitoring.For space buyers, it should be accessed through a new generation decision-making software, which would allow to automatically update in real time mediaplanning.For space sellers, it would allow to update in real time pricing of spacedepending on the audience and demand.It should be built to easily connect its data to advertising effectiveness measuringsystems. 41
  42. 42. My ideal system?The vision of the ideal system, for my radio, would cover a wider market thantoday to match my real targets (linguistic, geographical, lifestyle ...) and othertypes of media which match my targets and which could be complementary orcompetitors.Measures would be analyzed by any advertising and editorial target.It would include a mechanism for the space market (purchase and sale),powered by… Audience measurements: Cross media, reliable, targeted, continuous, available in real time, including audience share of all media likely to compete with me (no watermarking). Monitoring: advertising, cross-media, real-time and integrated to audience data. Structured data and detailed information on the broadcasts. Tools to assist decision making for both space buyers (optimizing media plans and impact assessment) and space sellers (optimizing prices).It would be really controlled. These checks would be continuously performed bytrue professionals of control.The whole system would be optimized, versatile, up to date and inexpensive. 42
  43. 43. How to start?These new systems will probably not be initiated by existing organizations forthe reasons already mentioned (not cross media, resistant to change, etc).They probably will be initiated by business clubs, cluster committees and incountries where new professional structures are still open et par trans-nationalmedia.Once they will begin to operate, even for small panels, they should triggerquestioning, awareness and widespread rapidly. 43
  44. 44. What we shall have to face 44
  45. 45. What we will have to confrontResistance of traditional mainstream media: who do not want to changeaudience measurement systems because they benefit them.The resistance of some institutes: The current audience measurement systemsare a too important a part of their turnover. Top managers wish to continue thisactivity, and for this reason, avoid changes they could not master and / orwhich would be less profitable.The inadequacy of existing professional groups: The new solutions may not besupported by today professional groups as they are currently organized bymedia type and by country. The new structure can form around the newreference unit: The advertising target. For example, one of these systems coulddevelop for luxury goods market. 45
  46. 46. Incompetence: A small portion of players has a global vision of the situationand real needs. Some do not have the skills to diagnose and others prefer nochange and "waiting to see more clearly.”Denial: some decision makers who actually have the information to understandthe strategic dilemma can not admit the threats it involve and wishes to hidereality. As they did previously decision makers in other sectors (telecom, micro-computer, vinyl disk, etc.).The stereotype of the consensus: it is an often heard sentence said byprofessionals and which would shudder in other industries: "The current system iscertainly imperfect, but it is consensus." For some players, the need of acommon rule, of a flat consensus is more important than the right strategy, thanquality and efficiency. 46
  47. 47. Preference for ambiguity: Some buyers and sellers prefer ambiguity whichmake impossible the evaluation of the quality of their services. They areaware of the limitations of their current efficiency, and in some cases,because their practices are not to the benefit of their clients.The weight of politics, sometimes: It happened in some countries thatprovided audience data were skewed for political reasons.The attempt to impose watermarking: when the need for change will bebinding for media research institutes, they will highlight systems that reinforcethem in the relations with the media, even if they are not technically the best.. 47
  48. 48. Thanks