Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

My concern about media audience measurements tamara silina - feb 26th 2013


Published on

A few months ago, I had a quite unclear understanding of what audience measurement is. After been involved in managing marketing for a new local radio station, I first looked at how decisions were made by sellers and buyers in the advertising space market. I was rather disappointed by what I found.
To summarize, I would say there are so many errors, bias and dysfunctions in traditional medias audience measurement that it seems it is very risky to take decisions based on this information.
Thank you to professionals in this field who answered my questions, who shared their knowledge and opinions with me through my blog and who read back my posts in despite of my ingenuous ignorance.
And sorry for my poor English.

  • Be the first to comment

  • Be the first to like this

My concern about media audience measurements tamara silina - feb 26th 2013

  1. 1. My concern about media audience measurments Tamara Silina, February 2013
  2. 2. A few months ago, I had a quite unclear understanding of what audiencemeasurement is. After been involved in managing marketing for a new local radiostation, I first looked at how decisions were made by sellers and buyers in theadvertising space market. I was rather disappointed by what I found.To summarize, I would say there are so many errors, bias and dysfunctions intraditional medias audience measurement that it seems it is very risky to takedecisions based on this information.Thank you to professionals in this field who answered my questions, who shared theirknowledge and opinions with me through my blog and who read back my posts indespite of my ingenuous ignorance.And sorry for my poor English. TamaraThanks to Orson Welles for the Citizen Kane few images I borrowed.
  3. 3. It is not aboutcreating panic, but to first question: are we inform… all going to die in horrible suffering? Patrick Chapatte, 28 février 2006, Le Temps.
  4. 4. What I heard from mediaWhat I have heard about media strategiesWhat purpose do audience measurements serve?New technical solutionsWhat we shall have to face
  5. 5. Here you are 5
  6. 6. What I have heard about old traditional media strategiesI selected three key ideas about what I was told about old media strategicanalysis of old traditional media groups. The share of the traditional media (radio, TV, print...) in the ad space market should decrease because of the inevitable decline of their audience. Ad expenses should be transferred mainly to digital media and other new areas of marketing investment. Their strategy would be to extend as much as possible the period during which traditional media can still be profitable.To do this, it would be vital for them not to disturb the factors that keeps theirbusiness especially audience measurement systems. 6
  7. 7. What I have heard about new and small traditional mediaNew entrepreneurs in traditional media business make different analysis: Today, It is much easier and less expensive to create and operate new traditional media with new production and broadcasting technologies and new content sources. New and small media would be more in synergy with Internet social networks. Their potential development would be higher because of their ability to deal with proximity, community, and generation issues through targeted and niches strategies.But, as far as their audience is not measured by the current audiencemeasurement systems, their development is strongly limited. 7
  8. 8. But what is the use of audience measurements? 8
  9. 9. What I understood about the purpose of audience measurmentAudience measurement is mainly used in the advertising space market ,to givea value to , and promote ad space. It also allows content managers to identifywho is listening, watching, reading, seeing what, in order to adjusts theircontents. But it is mainly the first function that determines the configuration ofaudience measurement.For advertisers and advertising agencies, audience measurement allows,before a campaign, to identify spaces with the best cost / potential impact ontheir target. During and after the campaign, it enables to evaluate somecampaigns performances. 9
  10. 10. In few words, how are audience measurement made?Who take decisions? Associations of media, agencies and advertisers decidewhich methods and means to implement. They choose, after bidding, researchagencies.How is it done? For radio and print, telephone or self-administered surveys areconducted. They may be continuous or by waves. Television audiencemeasurement use diaries surveys or people meters panels. For outdoor, they usesurveys or GPS tracking.Some few systems use automatic individual people meters (radio and TV).Who pays? These surveys and panels are mainly funded by the media whowant their audience to be measured. 10
  11. 11. Lie to me 11
  12. 12. Problems?In trying to understand these processes, I was very surprised by malfunctionsthat I did not expect to find in one of the first markets in the world economy.I discovered that most transactions are based on approximate data andtechniques that would have horrified brokers and traders in the fifties. 12
  13. 13. 17 key problems 1. Systems do not match the needs of data users, 2. Measures that are not cross - media, 3. Poor definition of the population whose audience must be measured, 4. Weak segmentation and profiling, 5. Irrelevant size and structure of samples, 6. Wrong survey techniques, 7. Bias in recruiting, 8. Question that induce untrue answers, 9. Irrelevant delay and frequency in data collection and delivery,10. Irrelevant presentation of delivered data,11. Delayed access to the data that make it obsolete,12. Non-compliance with specifications,13. Lax controls,14. Sidelining of the real experts,15. Dishonest data manipulation,16. Resistance of main media to change,17. Resistance of media research institutes to change. 13
  14. 14. 1. Audience measurements don’t match the needs of data usersThe first audience measurement function should be to help advertising spacebuyers and sellers, to setup space "value".To optimize and fluidify transactions in a market, the required information fordecision makers should be available by streaming and in real-time, throughconsistent tools for consultation, analysis, placement of order and post-evaluation.In all structured markets, these functions are integrated into homogeneous,optimized and controlled systems. In the advertising space market, thedata about programs and commercials monitoring, audience and forecasts,and the tools to exploit them are provided by heterogeneous suppliers withoutglobal vision of the users real needs, with virtually no control. Each of thesesuppliers thinks he is doing its best just by coordinating with others. This type oforganization produces a low result. 14
  15. 15. What current systems measure, is mainly the exposure of respondents orobserved members of panels, to what media broadcast. But is this concept ofexposure still enough to define the value of the advertising space?Don’t you think that, for example, involvement, interest, attention,appreciation, loyalty to programs, would be taken into account by theadvertisers and agencies if that data was available?Couldn’t we take into account the relationship between the consumers andthe products?The audience measurements are designed today without a global visionand without taking into account the real needs of those who determine theeffectiveness of advertising space and who develop the market space. 15
  16. 16. 2. No cross media audience measurementFor a potential buyer, the value of an offer advertising space, is related to histarget audience compared to those of all the alternative media and to theircomplementarily, for the target coverage.Media planners work is not only finding spaces that cost less by contact, but alsoidentifying space assemblages of complementary media to optimize therepetition on a specific share of the target. This is possible by spreading messagesthrough different types of media. Decision makers need to evaluate at the sametime the cost and complementarily of ad spaces of TV channels, radio stations,newspapers, etc.. This requirement has been explicitly expressed by internationalassociations of advertisers and, in 2009, by The Coalition for Innovative MediaMeasurement (, each year, new panels and new audience researches are setup without they taking into account this requirement. It seems that associationsand groups who control audience measurements do not understand the issue. 16
  17. 17. 3. Inadequate definition of what is to be measuredMore and more media are cross- national. For a radio or a television stationwhich broadcasts by satellite or on the Internet, the geographic area of itspotential audience is a linguistic territory. For example, Russian-speakingchannels are viewed in fourteen countries and Francophone channels, in fortycountries.Many advertisers who work in several countries have already internationalizedtheir branding positioning according to linguistic and cultural territory. Their adspace purchasing decisions are based on audience in these areas. To do this,some of them have to conduct their own multi-country measurements.International media as those who provide both satellite and cable ,have accessto inconsistent and heterogeneous audience measurement data. This is whytheir advertising space is under-estimated. 17
  18. 18. 4. Irrelevant segmentation and profiling toolsThe media audience is fragmented, it is more individual and independent. Newframeworks are needed to understand behavior and segmentation. Despite this,people who are recruited to participate in surveys and panels are brieflydescribed and profiled. In fact, in most cases, the minimum criteria is used,especially for television panels.Yet the media, advertisers and agencies communicate their targeting criteriaand segmentation. They need segmented data according to these criteria fortheir decisions. ( Page 5).Because they do not have such data, media planners are used to make riskyprojections and interpolating.A paradox: despite the fact that respondents are defined through only basiccriteria, it is generally very difficult and expensive to obtain data extractions usingthese criteria.What could drive those who designed these systems? Did they know thatrestricting samples qualification and profiling and reducing access to nativedata, would directly affect the media space demand? 18
  19. 19. 5. Irrelevant samples size and structureAll parts of a population are not worth the same as advertising targets. Wecould therefore expect that the targets who generate most ad expenses,would be covered by more accurate and more frequent measurements.But sample surveys and panels are most of the time structured as reducedmodels of population, whose audience has to be measured while they shouldover-represent segments of the population that are targeted by mostcampaigns.For this reason, a large part of the data is nearly never used.With this model logic, to get large enough useful sub-samples, it is necessarythat the size of the entire sample is very large, which ultimately would result inadditional - and hardly justifiable- costs.How is it possible that those who designed these systems, did not askthemselves what information was really useful? Why were they so anxious tohave very large samples, so unnecessarily representative of the population thatis useless to data users? 19
  20. 20. 6. Survey system which does’t match all segmentsThose who decide how audience measurement should be always want to use amonolithic methodology to cover the entire population, all segments, all profiles,the same way. This is probably in order to be able to compare audiencebehavior of different segments. These doesnt match any need of any data user.It is common sense that no data collection method can match all profiles. Someare more adequate to some segments and others to other segments.Those which can match for average profiles are not effective for higher socialcategories. For instance, television people meters are not able to measure theaudience of high purchasing power consumers, who, by the way, are subject toheavy advertising investments… 20
  21. 21. 7. Recruitment methods biasesI was lucky to be in touch with some media audience research teams. One ofthe things that disturbs them the most, is the very low representativeness ofpeople who are recruited for these measurements.One key factor is the low level of acceptance to participate in this kind ofsurvey or panel. A large part of the samples seems to be very difficult torecruit. For phone surveys, 90% refuse and 99% do for panels & people meters.It is very, very far below standards to get representative samples.Some profiles - especially in the higher social categories – are just impossible torecruit. This means that some key targets are just not represented in thesamples, even if it was in the scope of work and even if they appear in thesample description of the research institute.Contrary to what is sometimes said, no weighing method can correct theseproblems.Unfortunately, samples are widely made of atypical people who cannotrepresent real audience 21
  22. 22. 8. Questionnaires that produce wrong answersFor radio and print, most audience measurements are using self-administeredquestionnaires or are administered by phone or Internet.For radio, several researches have shown that a large part of peopleanswering, often don’t know what radio station they are listening to. And alarger part of them don’t remember what stations they had listened to the daybefore . At the same time, the most used information collected by thesesurveys is the last day audience.It is also shown that respondents tend to say the names of stations with a strongawareness or reputation, rather than to say that they do not remember. Thisgreatly benefit the most well known radio stations. Some of the respondentsalso say the names of the radio stations for which they have sympathy. Similarphenomena are observed for newspapers and magazines audience.,%20diversity,%20and%201a.pdfThe result is that these survey methods induce wrong answers and more often,media audiences which are better known ,are overvalued at the expense ofothers. 22
  23. 23. 9. Dysfunctions due to delays and frequencyRadio and print audience measurement surveys are conducted four or fivetimes a year in most countries. They need four or six week to be conducted anddata are provided with a delay of several weeks. However, most campaigns lastno more than three weeks.How could - we justify the quality of the space we sell with such delays? Imaginea buyer who will receive information on the effectiveness of his campaignsseveral months after? Some advertisers have to conduct their own audiencesurvey to be able to adjust their campaigns.It is sometimes said by some experts about radio and press audience thataudience vary little. But it is not true, as shown by researches using daily surveys(Arbitron, Radio audience measurement in New York using PPM). 23
  24. 24. 10. Irrelevant data access and deliveryData supplied by research institutes that produce audience measurementscannot be accessed through query tools that would cross numerous criteria.However, such software are available.According to some professionals, it appears that direct access to data wouldmake it easy to detect inconsistencies, oversized weighing and too lowrepresentation of some profiles in the samples. 24
  25. 25. 11. Information are too old when deliveredIn an ordinary market place, the more and the faster information is available,the stronger is the market active. In most markets, everything is done to get realtime data. Thats what I thought until I worked for a radio station.In most radio stations, like the one I worked for, audience measurements aremade using a telephone survey. About a thousand people are interviewed oneverything they heard the day before. One might say that data every week, isnot so bad and it could stimulate the market. But it is not the case. The data isprocessed once per quarter and it takes almost six weeks to have access to theearliest data.So by the end of January, I have data from November. I cannot use the datato justify my new year prices. Advertisers cannot make a decision and they donot buy my space. Why not treat and deliver data in a timely fit? 25
  26. 26. 12. Non-compliance with tenders specificationsThe impossibility to reach norms, poorly controlled recruitmentspecifications and data adjustments, drive institutes to use solutions thatdegrade the quality.Some profiles, for which recruitment is too difficult or too expensive, arereplaced by people out of the quotas: people of higher socio economic classare replaced by lower class; people in deep rural areas are replaced by nearcity countryside people, or suburban individuals, etc…Some abusive weighing complements abusive recruitment. Subsamples ofdifficult to recruit profiles, are over-weighed well beyond what can bestatistically acceptable. Some people may be counted ten times.These practices are the main reason for the barriers that are erected to accessindividual data from surveys and sorting purposes. They would make themvisible.We should be wary of audience measurement systems which do not giveaccess to individual data to any user, even for reason of consistency check. 26
  27. 27. 13. Lax controlsIn modern markets places such as the Stock Exchange, controlling systems arevery stringent. They focus on the design of systems, their operation, their integrityand they check absence of abuse in the use of data. Each of these controls isassigned to a separate institution whose activity is only controlling and auditing.For audience measurements, the practices are much less stringent. These controlsare organized through committees in which members have participated in thechoice of the institute, or are involved in audience measurement operations.The assessments are rather assigned to individual experts whose methods maylack rigor and whose independence is not validated. Often, these experts haveworked in audience measurement leading institutes. In addition, these controlsare very infrequent.Sometimes experts deliver two versions of their report: one for wide distribution,with very little criticism, and another for the committee, which is much morecritical. In these cases, it shows who the final client of audience measurements is. 27
  28. 28. 14. Real experts sideliningExperts who are requested by associations, for the design and control ofaudience measurement systems, have, most of the time, been involved inoperating this kind of survey or panel. They are not scientists, they aretechnicians. They just reproduce what they know, that is to say, the oldsystems, and endorse the currents practices.However, there are researchers who work and publish on these topics. Ifound many papers that addressed the problems I have identified. Theseresearchers are never asked to advise or for technical auditing.Researchers that I could speak to, are pessimistic about change inaudience measurement systems, because they think the currentorganizations that drive them, do not match the new required systems andthey will not be as revolutionary as they should be today. 28
  29. 29. 15. Intentional alteration of dataMore serious than the dysfunctions which are induced by aberrantspecifications, in some places, data is sometimes intentionally altered, that is tosay, deliberately changed. This happened at the request of media, contentproducers or agencies.Owners of media conglomerates explained to me how they had to pay to besure to get good audience figures.Some engineers who programmed software for media audience measurement,explained to me how audience of some channels can be routinely boosted bya discrete weighing.Moreover, some media planners have detected daily identical ratios for sometime - slots between the main channels, and for long periods.All these examples are recent.These practices may exist because the controls are really insufficient.because the controls are really insufficient. 29
  30. 30. 16. Leading medias resistance to changeGroups and committees that lead audiences measurements are oftendominated by mainstream media. They manage to impose themselvesbecause they have an historical leadership and because they pay more. It istherefore difficult to oppose their views.The orientations of these mainstream media are naturally more conservative,because they believe the relation between share of spend (ad expenditure)and share of voice (share of audience) and current audience measurementsystems protect their share of voice dominance.The subscription amount for a media to be measured, is often used by thesemainstream media as a barrier to exclude small media groups.It is clear that if audience measurement were cheaper, the leadershipof mainstream media would be weakened. That’s why they are fonder ofheavy methodologies (with large samples) and they denigrate innovations thatcould reduce cost. 30
  31. 31. 17. Leading institutes resistance to changeAudience measurements are highly profitable and not risky for leading researchagencies. In this time of crisis for the marketing research market, they are seen asa very stable part of the business.A contract for audience measurements generally last several years, it leaves anover 30% margin and probability of renewal is high. Few industries have bothsuch a level and so low a risk.For television audience measurement, there is a technical barrier to entry whichmakes IT virtually impossible to win A tender for a challenger institute.In some cases, major media are participating in the capital share holding ofinstitutes who measure their audience. This reinforces their conservatism. In othermarket places, such a conflict of interest would be prohibited.Sometimes, media research institutes are not afraid of conflicts of interest, andmay sell services to the media. "It is as if a manufacturer of thermometers soldsome aspirin." (E. Leser Slate. Http:// institutes will not add pressure to break the status-quo. They would fightagainst new technologies that could reduce the cost of media research andopen the door to new competing institutes. 31
  32. 32. New technical solutions
  33. 33. New available systemsThese are the Arbitron PPM, The GfK MediaWatch, Ipsos MediaCell, INVENTIVETechnologies and maybe others I have not yet identified.All of them can be used for both television and radio. They are portable andindividual.Most of them use watermarking technology. They can measure only theaudience of channels whose signal include a special code.Most are not integrated into overall systems of information and analysis for thepreparation of decision makers (buyers and sellers of ad space). 33
  34. 34. Arbitron PPMTThis Portable People Meter is a small box like a pager. It uses watermarking. Someversions could provide real time. Versions currently used transfer their data once aday.The fact that it is a case to carry always with you, may be a handicap. "Its a bit likethe electronic bracelet of custody" as described by one panelist.This is a closed system (hard and software), highly protected by patents. Thus, itcan evolve very slowly, more slowly than technologies on which it relies, includingtelecom, Internet and recognition signals. For this reason, it gradually loses itsadvance.It has been sold to many concession contracts and has partnerships with majorinstitutions that have not used it a 34
  35. 35. GfK MediaWatchIt is also a passive individual mobile people meter, integrated into a watch.Its main advantage is that it is constantly kept by the panelist.It does not provided in real time data because it is not connected. The panelistmust daily or weekly connect on a rack to upload data.A module can integrate the newpapers and magazines audiencemeasurement (declarative).It also useS watermarking, in spite the fact Gfk-Telecontrol had pushed farenough research in recognition of media signals without watermarking.GfK Telecontrol did not take into account that a watch is a very personal object,which carries the image and the style of the person who wears it. Theydeveloped quiet an ugly and very recognizable watch. This bias is a stronghandicap for MediaWatch panel recrutement.It is also a closed system (hardware and software) and therefore it can evolveonly slowly while the technological environment is changing 35
  36. 36. Ipsos MediaCellThis is an app for recognition of sound signals with watermarking, which can beincorporated in smart phones. It transmits in almost real time.This device would be ideal but the watermarking algorithms don’t have thecapacity for recognition of low level sound neither in noisy environment(discussion, engine noise, etc.).This is not a hardware closed system like the two previously mentioned systems. Itwill benefit other phone features such as localization, audience measurementbehavior of mobile Internet. This is one of the systems most likely to 36
  37. 37. INVENTIVE TechnologiesThis system also uses panelists smart phones and tablets. It is the only systemwhich doesnt use watermarking.The recognition technology for TV channels and radio stations can distinguishtheir signal from strong ambient noises. It can measure the audience of anychannels and any radio stations without limitation. It measures audience share, itprovides audience data in real time (2-3 minutes). It includes a module formeasuring print audience (declarative).It is integrated into a larger system with advertising monitoring tools and 37
  38. 38. Digression on watermarkingThe use of this technology requires each station / channel that wants to measureits audience to incorporate a numeric code in the audio signal. It does notmeasure audience of unmarked channels and stations.Audience share, which is known to be the first indicator used for contentaudience analysis cannot be measured.It is almost never mentioned that the watermarking signal recognition ismalfunctioning for the radio in noisy environments (such as in cars, which is one ofthe contexts in which the hearing of terrestrial radio is still virtually not challengedby Internet).The few independent technical tests conducted on traditional television peoplemeters (which use watermarking) have an average error of 10%. Tests even rareron passive people meters for radio rated more than 40% error. (Joe way marketing research institutes insist on watermarking is surprising when youknow its low performances. The reason for this preference could be that it forcesthe media to pay to be measured.The individual passive audience measurement can be performed withoutwatermarking and improved performance. 38
  39. 39. What to do for my radio? 39
  40. 40. What theory says?”Decision makers prefer alternatives with less uncertainty. Information reducesuncertainty. So buyers and sellers prefer bids supported by the latest, the mostrelevant, the most exhaustive, the most reliable, the most continuousinformation and in larger quantities”C. Shannon (1948) « A Mathematical Theory of Decisions and Communications ».That’s on of the key reasons why advertising space buyers divest traditionalmedia: the offer is not supported by this relevant information. Audiencemeasurements and buying and selling space systems them are handicaps forthe development of this market. It has to change; 40
  41. 41. What is said by the most advanced professionnals?The required system to replace those currently operating is clearly outlined bythe Coalition for Innovative Media Measurement and by experts.It should measure audience behavior by the same source (single source) and inreal-time, on and by any channel, for television, radio, Internet, print andoutdoor. It should include real time advertising monitoring.For space buyers, it should be accessed through a new generation of decision-making software, which would allow to automatically update in real time mediaplanning.For space sellers, it would allow to update in real time pricing of spacedepending on the audience and demand.It should be built to easily connect its data to advertising effectiveness measuringsystems. 41
  42. 42. My ideal system?The vision of the ideal system, for my radio, would be cover a wider market thantoday to match my real targets (linguistic, geographical, lifestyle...) and othertypes of media which match my targets and which could be complementary oreven competitors.Measures would be analyzed by any advertising and editorial target.It would include a mechanism for the space market (purchase and sale),powered by… Audience measurements: Cross media, reliable, targeted, continuous, available in real time, including audience share of all media likely to compete with me (no watermarking). Monitoring: advertising, cross-media, real-time and integrated to audience data. Structured data and detailed information on the broadcasts. Tools to assist decision making for both space buyers (optimizing media plans and impact assessment) and space sellers (optimizing prices).It would be really controlled. These checks would be continuously performed bytrue control professionals.The whole system would be optimized, versatile, up to date and inexpensive. 42
  43. 43. How to start?These new systems would probably not be initiated by existing organizationsfor the reasons already mentioned (not cross media, resistant to change, etc).They probably would be initiated by business clubs, cluster committees and incountries where new professional structures are still open and by trans-national media.Once they will begin to operate, even for small panels, they should triggerquestioning, awareness and widespread rapidly. 43
  44. 44. What we shall have to face 44
  45. 45. What we will have to confrontResistance of traditional mainstream media: who do not want to changeaudience measurement systems because they benefit from the existing ones.The resistance of some institutes: The current audience measurement systemsare a too important a part of their turnover. Top managers wish to continue thisactivity, and for this reason, avoid changes they could not master and / orwhich would be less profitable.The inadequacy of existing professional groups: The new solutions may not besupported by today professional groups as they are currently organized bymedia type and by country. The new structure can form around the newreference unit: The advertising target. For example, one of these systems couldbe developed for the luxury goods market, for pregnant future mothersmarket… 45
  46. 46. Incompetence: A small portion of players has a global vision of the situationand real needs. Some do not have the skills to diagnose and others prefer nochange and adopt a "wait and see” attitude.Denial: some decision makers who actually have the information to understandthe strategic dilemma can not admit the threats it involves and they wish tohide reality, as did previous decision makers in other sectors (telecom, micro-computer, vinyl disk, etc.).The stereotype of the consensus: it is an often heard sentence said byprofessionals and which would make you shudder in other industries: “ Thecurrent system is certainly imperfect, but it is consensus." For some players, theneed of a common rule, of a flat consensus is more important than the rightstrategy, quality and efficiency. 46
  47. 47. Preference for ambiguity: Some buyers and sellers prefer ambiguity whichmake impossible the evaluation of the quality of their services. They areaware of the limitations of their current efficiency, and in some cases,because their practices are not to the benefit of their clients.The weight of politics, sometimes: It happened in some countries thatprovided audience data were skewed for political reasons.The attempt to impose watermarking: when the need for change will bebinding for media research institutes, they will highlight systems that reinforcethem in the relations with the media, even if they are not technically the best.. 47
  48. 48. Thanks