Evaluating the Effectiveness of your Media Relations Strategies Media Relations in the Digital Age Conference Jon Wade – Head of Digital, Asia Pacific Weber Shandwick
I’m just back from scuba diving in the Philippines…
Measurement is pretty crucial in scuba diving: It tells you when you’re about to run out of air It tells you if you’re likely to get nitrogen narcosis, oxygen toxicity or ‘the bends’ It essentially stops you dying!!
Whilst, PR measurement isn’t quite in the same life-or-death category, nor the most fascinating subject matter on the planet… …it’s still a vital component of our communications activities because it allows us to measure and optimise the effectiveness of our activities
Contents Introduction Barcelona Principles Measurement Framework Prioritising Measurement Measurement Tools A bit of fun to end… Q&A
Association for Measurement & Evaluation of Communications Institute for Public Relations “Barcelona Principles” for PR measurement
7 ‘Barcelona Principles’ Importance of goal setting and measurement Measuring the effects on outcomes is preferred to measuring outputs The effect on business results can and should be measured where possible Media measurement requires quantity and quality AVEs are not the value of Public Relations Social Media can and should be measured Transparency and replicability are paramount to sound measurement
Principle #1: Importance of goal setting and measurement
Goal-setting and measurement are fundamental aspects of any public relations programme
Goals should be as quantitative as possible and address who, what, when and howthe PR program is intended perform
Implication: If this is true, we need to invest both time and budget in the goal setting and measurement aspects of our programme
A couple of acronyms that can help us with Principle #1 Regarding our objectives: S.M.A.R.T. Regarding our communication goals: R.E.A.
Our objectives should be: pecific easurable chievable ealistic imed S. M. A. R. T. Are our objectives clearly defined? Is it possible to measure the objectives? Are the objectives even possible? Do we have the resources / time to deliver? What is the timescale?
Are our communications primarily concerned with generating…? Reach Are we trying to increase product / brand awareness? Engagement Are we trying to increase product / brand consideration or willingness to buy Action Are we trying to increase product sales?
A perfect client brief (from a measurement point of view) We’re launching our new <insert product / brand>. The key business objective for the PR campaign is to raise awareness amongst <insert target audience> from the current 20% awareness level to 30% within 6 months as measured by an online survey. ☐ ☐ R.E.A. S.M.A.R.T.
Principle #2: Measuring the effect on outcomes is preferred to measuring outputs
Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions with respect to a company, NGO, government or other entity.
Practices for measuring the effect on outcomes should be tailored to the business objectives of the PR activities.
Quantitative measures preferred
However, qualitative methods to supplement
Measures & Tools
But of course it’s not always possible to measure everything… …so we need a way of priortising our measurement programme.
One method: 1 Pick one primary business objective only
Let’s take the example where the key business objective we are primarily trying to manage through our media relations strategy is: Improving Brand Reputation
And based on our earlier framework, to achieve this objective, our tactics and measurement are going to be primarily concerned with driving engagement with consumers: Having them engagewith the brand online and share / recommend the brand to people within their network Having their opinion of the brand become more positive Having them use positive language in their conversation about the brand online
If these are our objectives, we would consider the following measures: The Brand’s Net Promoter Score Opinions of the brand expressed online Sentiment of online conversations involving our brand The exact methods we select will depend on the associated costs and budgets available for measurement
Net Promoter Study Q: “How likely are you to recommend this brand to your friends and family?” NPS = % Advocates - % Detractors (0 – 10 scale; 0 – 6 Detractors; 7 – 8 Neutral; 9 – 10 Advocates) Reputation (NPS) Long-term Profitability & Growth
Engagement – NPS Correlations
Engagement – NPS Trends & Benchmarks
Engagement – Reputation Survey
Engagement – Opinion, Crimson Hexagon
Principle #3: The effect on business results can and should be measured where possible
To measure business results from consumer or brand marketing, models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables, are a preferred choice. Related points are:
Clients are creating demand for market mix models
The PR industry has to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR
The PR industry needs to develop PR measures that can provide reliable input into market mix models
Marketing Mix Models SALES = a + b[TV$] + c[PR$] + d[DM$] + e Where: a = ‘Natural’ Sales b = Ratio of TV spend to incremental sales c = Ratio of PR spend to incremental sales d = Ratio of DM spend to incremental sales e = margin of error
How can you split out the effects of PR for this type of data?
Online techniques providetools to do this for digital PR Example: We can drop a custom cookie to the audience who visits our Facebook properties Cookie = 0 Cookie = 1 Control Exposed By surveying a sample of those that have a cookie vs those that haven’t we can split out the effect of our Facebook efforts.
Principle #4: Media measurement requires quantity and quality Overall clip counts and impressions are usually meaningless. Instead, media measurement, whether in traditional or online channels, should account for:
Impressions among the stakeholder or audience
Quality of the media coverage including:
Credibility and relevance to the stakeholder or audience
Inclusion of key messages
Inclusion of a 3rd party or company spokesperson
Quality can be negative, positive, or neutral
Automated sentiment measurement allows us to to get monitor the quality of the coverage we are getting as well as early warning of potential issues and crises
Firebell is an online crisis simulation programme that allows Weber Shandwick client’s to practice responding to an online crisis in a safe offline environment
Reach – Klout
Reach – SNS
Reach – DoubleClick AdPlanner
Principle #5: AVEs are not the value of Public Relations
Advertising Value Equivalents (AVEs) do not measure the value of public relations and do not inform future activity; they measure the cost of media space and are rejected as a concept to value public relations
Where a comparison has to be made between the cost of space from earned versus paid media, validated metrics should be used, stated for what they are, and reflect:
negotiated advertising rates relevant to the client, as available
quality of the coverage, including negative results
physical space of the coverage, and the portion of the coverage that is relevant
Multipliers intended to reflect a greater media cost for earned versus paid media should never be applied unless proven to exist in the specific case
The problem with AVEs There is no research to back-up the claim that a news story and an advertisement have an ‘equivalent’ effect News media credibility varies by story (positively & negatively) Has been dropping in past 2 decades Often the goal of PR is to reduce publicity…this cannot be measured using AVEs Ads are usually designed to work at frequency, news stories tend to be relatively unique There isn’t always an equivalent advertising rate – front covers, coverage during news bulletins with no ad breaks, coverage on sites that don’t accept advertising etc There is no advertising equivalent of a ‘bad’ or ‘unfavourable’ story – should we ignore those?
Measuring Worth 47 Media, Blogger & Influencer Interviews 94 Activities Facebook, YouTube, Blog & Twitter Posts 170 Earned & Social Media Placements 3.9M Reach Earned & Social Media Impressions 64% Earned & Social Message Penetration 27% Relevance Earned & Social Media Share 14% Increase in Brand Engagement 27% Outcomes Category Sales Share $4.72 Earned CPM (Cost Per 1K Impressions) $8.22 Worth Social CPE (Cost Per Engagement)
Earned CPMs Total Cost of Earned Coverage X 1000 Total Earned Impressions Cost Per Engagement Total Cost of Earned Engagement Total Engagements
Engagement – PostRank
Principle #6: Social Media can and should be measured
Social media measurement is a discipline, not a tool; but there is no “single metric”
Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods
Evaluating quality and quantity is critical, just as it is with conventional media
Measurement must focus on “conversation” and “communities” not just “coverage”
Understanding reach and influence is important, but existing data sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success
Engagement – Google AdWords
Engagement – Google Trends
Engagement & Action – Web Analytics
Principle #7: Transparency and replicability are paramount to sound measurement PR measurement should be done in a manner that is transparent and replicable for all steps in the process, including specifying: Media Measurement:
Source of the data (print, broadcast, internet, consumer generated media) along with criteria used for collection
Analysis methodology – for example, whether human or automated, tone scale, reach to target, content analysis parameters
Methodology – sampling frame and size, margin of erroretc
Questions –should be released as asked (wording and order)
Statistical methodology- how specific metrics are calculated
ARROW: Flexible, Integrated Measurement Framework Activities Reach Relevance Outcomes Worth What activitieswere performedto achieveresults? Did you reachyour audience?How manyimpressions,web visits,reports,attendees, etc. weregenerated? Were yourelevant to youraudience? Were you credible? Did your ideas and messages resonate? Did you drive conversation? What business results did you achieve? Awareness? Engagement? Reputation? Leads? Sales? Loyalty? Advocacy? What is the estimated value of your communication efforts? Quantity/Output Quality/Outtakes Business Impact Value/Efficiency Communications Team Marketing Team Executive Team
Light ARROW Scorecard 47 Media, Blogger & Influencer Interviews 94 Activities Facebook, YouTube, Blog & Twitter Posts 170 Earned & Social Media Placements 3.9M Reach Earned & Social Media Impressions 64% Earned & Social Message Penetration 27% Relevance Earned & Social Media Share 14% Increase in Brand Engagement 27% Outcomes Category Sales Share $4.72 Earned CPM (Cost Per 1K Impressions) $8.22 Worth Social CPE (Cost Per Engagement)
Full ARROW Scorecard
OK – hopefully that wasn’t too dull… To end – a bit of fun…
Data Visualisations Data doesn’t have to be boring… …in fact the way you represent the data can make the difference between whether your audience engages with it or not.
Jonathan Harris and Sep Kamvar – We Feel Fine
Thanks for listening.
Any Questions? Jon Wade Head of Digital, APAC – Weber Shandwick +852 9789 8973 @digital_apac firstname.lastname@example.org