Barcelona declaration of measurement principles final with results voting. 20.06.10

Barry Leggetter
Executive Director AMEC
Summit Director

Creating the Barcelona Declaration of Measurement Principles
Moderator
David B. Rockland Partner/CEO Ketchum Pleon

Barcelona Declaration of Measurement Principles Global Alliance ICCO Institute for Public Relations Public Relations Society of America AMEC U.S. & Agency Leaders Chapter

Why Are We Here? For the Common Good of Our Industry.
The communications landscape is changing rapidly
A lack of clear standards and approaches to PR measurement results in the profession not always being taken seriously; it doesn’t count unless you can count it
AVEs, random use of multipliers, and other silly metrics and practices diminish the integrity of the profession and the market size
As a profession, we spend more time arguing amongst ourselves rather than building the field
We are at a moment in time to make difference, and alter the path we are on, taking advantage of the opportunity to “own” the new communications environment

New AMEC research shows strong demand for AVEs
Q8 Current trends in the measurement and evaluation market (base: 34)
Chart does not include % saying “neither agree nor disagree”
Proportion disagreeing 6%

Increased Interest in International Measurement and Social Media; ad hoc programmes on the decline
Q8 Current trends in the measurement and evaluation market (base: 34)
Chart does not include % saying “neither agree nor disagree”
Proportion disagreeing 6% 47% 3%

Towards an evaluation standard for marketing PR – market mix modelling Speaker Richard Houghton President ICCO on behalf of New York University

Lou Capozzi Immediate Past President, ICCO and Professor, New York University

Team of graduate students from NYU’s Masters Degree program in public relations and corporate communications
Supervised by Prof. Capozzi
Gained a grant from NYU and support form the International Communications Consultants Organisation
Broad review of the literature
Goal – seek commonalities
NYU Student Audit

Large PR firm models
Country models
UK, Canada and Germany
PR measurement vendor models
Advertising models
Market mix modelling vendors
NYU Student Audit - Areas Covered

Real need and appetite for a standardise approach
Enormous breadth and diversity of existing approaches
Variation is too great to be rationalised into a single model
All models have a common flaw
NYU Student Audit – Conclusions

Broadly adopted by large-scale marketers
Readily available for PR to connect to
Enables comparisons between all the elements in the marketing mix
Fits with many existing PR evaluation techniques
Why market mix modelling?

Focused tightly on marketing PR, especially for large-scale programs
Vendor neutral
Strikes an effective balance between cost and credibility
Presented in terms familiar to marketers
Practical with low barriers to entry
Model requirements

Total PR program costs (negotiated rates)
Markets covered
Circulation/audience reach (no multipliers)
Impressions
Tone/Quality score
Impact score
Potential Data Elements

Up front commitment required
Historical data needed for MMMs
Weekly tracking generally required
Larger PR investments required to show impact
Budgets need to be tied to PR program elements
Challenges

Meet with thought leaders
AMEC US and agency research leaders
Corporate pioneers
Meet with MMM vendors, in-house operations
Propose a standard
Get AMEC support
Seek support from major PR associations
Promote
Next Steps

John Paluszek Chair Global Alliance For Public Relations and Communication Management

Mission: To raise standards and promote public relations globally
Mantra: "We can -- and we must -- all learn from each other"
Members: National PR associations in 70 countries and leading international PR associations:
Serving 175,000 practitioners and educators.
Wide range of services: "Landscapes" of PR in 22 countries . Global PR education study
The Global Alliance For Public Relations and Communication Management

Signature Event The Stockholm Accords

" Stockholm Accords" Measurement Opportunities
Effectiveness of public relations communications/messages
Credibility of source
Credibility of content
Familiarity of content
Long term effectiveness of "Stockholm Accords"
Integration of public relations into university management curriculum
Appointments of public relations professionals in C-suites
Improved respect for the profession in business and general media

The Consultancy View Programme Measurement Principles Richard Houghton

International Communications Consultancy Organisation
Umbrella organisation for 1500 PR agencies in 28 countries
Our members are PR trade associations
Remit:
Standards – CMS in 15 countries
Forums – senior management
Insight – World Report
Huge variety in the maturity of individual markets

ICCO Members’ Views

The Consultancy View Principles need to be…
Based on straight forward and easily understood methodology
Needs to be understood and used by all account handlers at all levels
Applicable across PR disciplines
Structured to allow for growing digital and social media communications

The Consultancy View We like to think evaluation differentiates
Many agencies feel that their proprietary evaluation processes give them a competitive edge
We may, of course ,be kidding ourselves!
If clients adopt the principles, consultants will too

The ugly reality AVEs work for the finance guy (mostly)
No they don’t give provide insight into the effectiveness of a programme or campaign
No you can’t really compare advertising with PR
No they don’t take in account tonality or stance
No they are no use at all for the vast majority of online activity
But….
… if finance guys want them consultancies are likely to provide them

Creating the Declaration of Measurement Principles
Institute for Public Relations Commission on Public Relations Measurement and Evaluation
Pauline Draper – Chair, IPR Commission

The Commission Who are we and what do we do?
The Commission is unique in that it comprises some of the leading thinkers from
academia
corporate/government/not-for-profit
agency
research provider
Our purpose
to establish best practice for PR research and measurement
to issue authoritative White Papers
to promote excellence in PR research and measurement through the Jack Felton Golden Ruler Awards and partnering with others

Example White Papers Available from InstituteforPR.org
A primer on measuring corporate reputation: a case study on the Dow Chemical Company
Doing Measurement Right: One Organization’s Experience Creating a Best-In-Class Measurement Program from Scratch
Isolating the Effects of Media-based Public Relations on Sales: Public Relations Optimization Through Marketing Mix Modeling
Using Web Analytics to Measure the Impact of Earned Online Media on Business Outcomes: A Methodological Approach
A New Paradigm for Media Analysis: Weighted Media Cost
Guidelines for Setting Measurable Public Relations Objectives: An Update

Definition: The calculation of space or time used for earned media (publicity or news content) by comparing it to the cost of that same space or time if purchased for advertising.
AVE is used because it assigns a monetary value to public relations using a metric that is simple to understand, simple to calculate, and uses data that is readily available.
AVE suggests that the space/time occupied by earned media is equivalent to the same space/time purchased
AVE is not a proxy for measuring public relations’ ROI
Advertising Value Equivalents Why not? Outcomes rather than Outputs or Outtakes

Measurement and evaluation practices should demonstrate the effectiveness of public relations to help meet organizational goals.
Measuring the quality of public relations output is necessary in order to demonstrate its contribution to reaching these goals.
Any measure of media coverage should include variables such as tone , prominence , placement , appearance of key messages , the portion of story that applies to the organization or its key messages, and the credibility and targeted reach of the medium in which the message appears .
Whenever possible, it is best to link media coverage with expected outcomes . These outcomes should also be tied to organizational goals.
Advertising Value Equivalents Measuring the Value of this Marketing Communication

Measuring OUTCOMES
Benchmarking:
To past performance
To closest rival
To competitors
Map outputs to other metrics:
Share price
Sales
Web analytics data
Specific research:
Pre/post survey
Message testing
Journalist audit
Utilize other data:
Surveys
Focus groups
Marketing mixed modeling / econometric modeling
Client and Employee satisfaction studies
Practices for taking measurement further

Barcelona Declaration of Measurement Principles Public Relations Society of America Gary McCormick Chairman and CEO

As a first step, ensuring that four basic questions are asked regarding the work that is to be undertaken:
Whom are you seeking to affect?
What about them are you seeking to affect?
How much must they be affected to be successful?
By when does this effect need to occur?
The Business Case for Public Relations A framework for building measurement into public relations programs – a lexicon and recommended approaches.

Financial
Generates Revenue, Sales, Profit; Marketing public relations drives sales.
Investor public relations drives investment, valuation.
Public relations drives donations and membership for relevant organizations.
Reputation / Brand Equity
Increases likelihood to purchase / consider your brand(s).
Minimizes the effects of a crisis and rebuilds trust. Establishes credibility
Employees and other Internal Publics
Increases employee satisfaction and engagement, leading to greater efficiency, increased retention, reduced turnover, lower recruiting costs and higher productivity.
Public Policy
Creates public awareness, understanding and support for legislation, regulation and political candidates.
Affects voter behavior. Helps pass legislation, regulation and initiatives.
As a second step, connect the desired outcomes to strategic goals – one of four key areas:

Financial
Consumer response: Field consumer survey; determine purchase levels and exposure to public relations results; isolate causal effects through statistical analysis.
Market Mix Modeling / Econometric Modeling: Gather public relations output / outcome data by function, market, region and time period; factor revenue-generation by market, by region and over time. Apply regression analysis.
Reputation / Brand Equity
Benchmark reputation / relationship metrics via survey prior to a campaign; repeat every three to six months.
Correlate attitudinal studies with customer purchase attitudes and behavior.
Employees and other Internal Publics
Compare control groups to employee populations exposed to public relations activities.
Focus on performance outcomes, not attitudes or awareness, such as employee satisfaction and engagement findings, employee turnover statistics and other recruitment data.
Public Policy
Use tracking services to track awareness, correlate to public relations activity, and connect to actual votes recorded and legislative outcomes.
Third, measure the desired outcomes not in terms of outputs – clips or activity – but in terms of outcomes:

Barcelona Declaration of Measurement Principles AMEC U.S. & Agency Leaders Chapter Tim Marklein Executive VP, Measurement & Strategy Weber Shandwick

AMEC’s first international chapter. Launched in fall 2009. Provides a networking forum for U.S. research and agency leaders. Represented on AMEC Board by inaugural chair David Rockland of Ketchum. Includes 16 members: Existing AMEC Members: BurrellesLuce, Cision, Dow Jones, Echo Research, Report International, VMS Large Agency Research Leaders: Edelman/StrategyOne, Fleishman-Hillard, Hill & Knowlton, Ketchum, MS&L, Ogilvy, Weber Shandwick Midsize Agencies and Specialist Research Firms: Chandler Chicco, Evolve24, Leading Communicators U.S. & Agency Leaders Chapter

Topic #1. Ad Value Equivalency
Building on IPR Commission’s Oct’09 vote to “condemn the name, concept and practice of AVEs,” we agreed to the following fundamental principles:
This topic is germane to AMEC.
Outcome based metrics – attributes, behavior, business results, etc. – should always be used in preference to output metrics, whenever possible.
When measures are based on cost of media space or time, they should be referred to as “costs” not “values.”
If media costs are used, they should be adjusted for quality (tone, message, etc.) and for the actual space or time occupied by a client or competitors.
Multipliers should not be used unless validated by third-party audits, surveys or market mix models.
Negotiated rates should be used (not rate cards).
Methodologies must be transparent and repeatable.

Topic #2. Social Media Measurement (part one of two)
Building on AMEC U.S. member experience, we agreed to the following fundamental principles:
Social media can be measured.
Social media measurement is a discipline and a process, not a tool.
There is no “single metric” for social media.
Media content analysis has a fundamental role in social media measurement – and should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.
Organizations need clearly defined goals and outcomes for social media in order to measure it.
Refer to PRSA “Business Outcomes” outline: http://comprehension.prsa.org/?p=628
Refer to Altimeter Group “Social Marketing Analytics” outline: http://bit.ly/dldIHf

Topic #2. Social Media Measurement (part two of two)
Building on AMEC U.S. member experience, we agreed to the following fundamental principles:
Evaluating quality and quantity together is critical, just as it is with conventional media.
Given the scale and volume of social media, technology-assisted analysis is necessary – but human reading and coding is still valuable for precision analysis of tone, messaging, nuance, etc.
Measurement must adapt to the media, focusing on “conversation” and “communities” not “coverage.”
Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable.
Experimentation and testing are key to success – though measurement and communication fundamentals still apply.

Topic #3. Market Mix Modelling
Building on the initial NYU market mix modeling report and AMEC U.S. member experience, we agreed to the following fundamental principles:
Clients are creating demand for market mix models to evaluate the impact of consumer marketing.
The PR industry needs to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches.
The PR industry needs to develop PR measures that can provide reliable input into market mix models.
Such measures might focus on media quantity and quality, messaging and audience impact, traditional versus social media, consistency with other marketing disciplines, etc.

Topic #4. Transparency
Clients are entitled to know how their content was obtained and the methodology used to append any metadata to that content:
Sources :
Print - Physical clippings, aggregator feeds, website links
Broadcast – closed caption text, speech to text, manual observation
Internet – aggregators, spiders, text scrapes, screen views
Blogs – main post only or post plus comments and forwards
Qualitative assessments :
Human or machine scored, sampled or all coverage
Tone – criteria for scoring should be clear and shared
Reach metrics – the source of the data and the criteria they are based upon should be clear, i.e. persons 2 + , persons 18 + , daily or monthly visitors, unique visitors, etc.
Other – content analysis of messages, scorecards, calculated measures such as impact scores or value scores should be understandable and not a “black box”

Networking Refreshment Break

A Declaration of Measurement Principles David Rockland Partner / CEO, Managing Director Ketchum AMEC Board Member IPR Measurement Commission, Past Chairman

TELE-VOTING SYSTEM

The debate: how to vote
You will be advised when to vote
You will be given clear options on which to vote
Please click your preferred option during the voting time
To change your option, click again. The last one you record will be the one accepted by the system
The result will be shown on the screen once all voting finishes
Summit staff will collect the voting keypads at the end of the conference
Further opportunity for comment to [email_address] until July 15

Our Summit Voting Test!
Q. Is Catalan the officially spoken language in Barcelona?
YES: Press 1
NO: Press 2

Seven Principles
Importance of Goal Setting and Measurement
Media Measurement Requires Quantity and Quality
AVEs are not the Value of Public Relations
Social Media Can and Should be Measured
Measuring Outcomes is Preferred to Measuring Media Results
Business Results Can and Should Be Measured Where Possible
Transparency and Replicability are Paramount to Sound Measurement

Principle 1:Importance of Goal Setting and Measurement
Goal-setting and measurement are fundamental aspects of any public relations program
Goals should be as quantitative as possible and address who, what, when and how much the PR program is intended to affect
Measurement should include representative traditional and social media; target audience changes in awareness, comprehension, attitude, and behavior as applicable; and business results

The Seven Principles
Principle 1
Importance of Goal Setting
and Measurement
YES: Press 1
NO: Press 2

Principle 2: Media Measurement Requires Quantity and Quality
Overall clip counts and general impressions are usually meaningless. Instead, media measurement, whether in traditional or online channels, should account for:
Impressions among the target audience
Quality of the media coverage including:
Tone
Credibility
Message Delivery
Inclusion of a 3 rd party or company spokesperson
Prominence
Visual dimensions
Quality can be negative, positive, or neutral

Principle 2
Media Measurement Requires
Quantity and Quality
YES: Press 1
NO: Press 2

Principle 3: AVEs are Not the Value of Public Relations
Advertising Value Equivalents (AVEs) do not measure the value of public relations and do not inform future activity; they measure the cost of media space
Where a comparison is to be made between the cost of space from earned versus paid media, validated metrics such as Weighted Media Cost should be used and reflect:
negotiated advertising rates
quality of the coverage (see Principle 2), including negative results
physical space of the coverage, and the portion of the coverage that is relevant
Multipliers intended to reflect a greater WMC for earned versus paid media should never be applied unless proven to exist in the specific case

Principle 3
AVEs are Not the Value of
Public Relations
YES: Press 1
NO: Press 2

Principle 4: Social Media Can and Should Be Measured
Social media measurement is a discipline, not a tool; but there is no “single metric”
Organizations need clearly defined goals and outcomes for social media
Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods
Evaluating quality and quantity is critical, just as it is with conventional media
Given the scale and volume of social media, technology-assisted analysis is necessary – human reading and coding is valuable for precision analysis of tone, messaging, nuance, etc.
Measurement must focus on “conversation” and “communities” not “coverage”
Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success

Principle 4
Social Media Can and Should
Be Measured
YES: Press 1
NO: Press 2

Principle 5: Measuring Outcomes is Preferred to Measuring Media Results
Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions, and other shifts in target audiences regarding a company, NGO, government or entity, as well as the audience’s own beliefs and behaviors
Benchmark and tracking survey research are the preferred practices for quantitative measurement
Standard best practices in survey research including sample design, question wording and order, and statistical analysis should be applied in total transparency

Principle 5
Measuring Outcomes is
Preferred to Measuring Media Results
YES: Press 1
NO: Press 2

Principle 6: Business Results Can and Should Be Measured Where Possible
Models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables that drive sales, are a preferred choice for measuring consumer or brand marketing. Related points are:
Clients are creating demand for market mix models to evaluate the impact of consumer marketing
The PR industry needs to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches
The PR industry needs to develop PR measures that can provide reliable input into market mix models
Survey research can also be used to isolate the change in purchasing, purchase preference or attitude shift resulting from exposure to PR initiatives

Principle 6
Business Results Can and
Should Be Measured Where Possible
YES: Press 1
NO: Press 2

Principle 7: Transparency and Replicability are Paramount to Sound Measurement
PR measurement should be done in a manner that is transparent and replicable meaning the detailing of:
Media Measurement:
Quantitative – source of the content: print, broadcast, internet, consumer generated media along with criteria used for collection
Qualitative – human or automated, tone, reach to target, content, overall media impact
Surveys:
Methodology –sampling frame and size, margin of error, probability or non-probability
Questions – all should be released as asked
Statistical methodology- how specific metrics are calculated

Principle 7
Transparency and Replicability are
Paramount to Sound Measurement
YES: Press 1
NO: Press 2

Next Steps: Panel Organizations Delegates AMEC

THE RESULTS Yes No 1. Importance of Goal Setting and Measurement 97% 3% 2. Media Measurement Requires Quantity and Quality 97% 3%
3. AVEs are not the Value of Public Relations
Do not measure the value of PR or future activity
Where comparisons made validated metrics should be used
Multipliers never be applied unless proven to exist
70% 92% 57% 72% 30% 8% 43% 28% 4. Social Media Can and Should be Measured 93% 7% 5. Measuring Outcomes is Preferred to Measuring Media Results 87% 13% 6. Business Results Can and Should Be Measured Where Possible 86% 14% 7. Transparency and Replicability are Paramount to Sound Measurement 95% 5%

Barcelona declaration of measurement principles final with results voting. 20.06.10

by Eric Schwartzman on Jul 09, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 3

Statistics

Barcelona declaration of measurement principles final with results voting. 20.06.10 — Presentation Transcript