2009 Measurement 101

Measurement 101 A Presentation to the IPR Measurement Summit October 2009 Katie Delahaye Paine Member, IPR Measurement Commission www.instituteforpr.com CEO [email_address] www.kdpaine.com

Why Measure?

“ The main reason to measure objectives is not so much to reward or punish

individual communications manager for success or failure as it is to learn from the

research whether a program should be continued as is, revised, or dropped in favor of another approach ”

James E. Grunig, Professor Emeritus, University of Maryland

“ If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous, capricious, motivated by innumerable conflicting interests, and conflicting desires.” Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

The Ah-Ha moment, the Lotus PR Report Percent of impressions containing messages by product

The Ah-Ha moment, the Lotus PR Report Cost per message communicated

Victory Over Martians Confirmed! Share of exposure over time 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Jan Mar May Jul Sep Nov Jan Mar May Jul Sep Nov Jan IN TI MO NS

Interviews and media advisories generated best coverage 0% 20% 40% 60% 80% 100% Application articles Contract wins Exec Interview Media advisory Release + conference Press release plus VNR Product review Industry issue Trade show/event No Message Negative Message Positive Message

Comparing the effectiveness of different tactics

Correlation exists between traffic to the ASPCA web site and the organization’s overall media exposure

Correlations also exist between online donations to the ASPCA and the organization’s overall media exposure

Facebook: Correlating MSM, CGM and signups

Strong correlation

Non-negative discussion only

A measurement timeline

10 signs that it’s the end of measurement as we know it

BMC Software measures communications effectiveness based on contribution to EPS

Procter & Gamble is now paying for engagement, not eyeballs

Sodexo measures success of its Twitter program by showing $300K reduction in recruitment costs

USO compares PR and PSAs to determine effectiveness of fund raising

HSUS measures Social media success by calculating $650,000 in new donations

Immunize BC counts calculates awareness + shots delivered

The Red Cross measures the effectiveness of Twitter via lives saved and harm avoided

IBM can calculate more sales from a $500 podcast than a $40K ad program

BestBuy measures 85% lower turnover as a result of its Blue Shirt community

Advertisers are starting to admit that all their measures are flawed

What do you need to measure to get to the Holy Grail of ROI measurement? Data.

Outputs?

Did you get the volume of coverage & visibility you wanted?

Did you produce the promised materials on time and on budget?

Outtakes?

Did your target audience see the messages?

Did they believe the messages?

Outcomes?

Did audience behavior change?

Did the right people show up?

Did your relationship change?

Did sales increase?

Goals, Actions and Metrics Goal Action Output Metric Outtake Metric Outcome Metric Increased on-line donations Revamp website Amount of content on web site % perceiving ASPCA as a reliable source % increase in web traffi and donations #1 most trusted source for information on companion animals Increase staffing and resources for communications Increased exposure of “trusted source” message Increased perception of ASPCA as trustworthy and comprehensive source % increase in agreement with the statement Website is preferred site for information Add content, features to web site, keep up to date % increase in traffic % agreeing with the statement # 1 rankings, and time spent on site Integration of department Reorganize department More integrated materials produced on time and on budget Internal perception is “one department” Consistent messaging throughout

The 7 steps to PR ROI

Define the “R” – Define the expected results?

Define the “I” -- What’s the investment?

Understand your audiences and what motivates them

Define the metrics (what you want to become)

Determine what you are benchmarking against

Pick a tool and undertake research

Analyze results and glean insight, take action, measure again

Step 1: Define the “R” what return do you expect? s

What were you hired to change?

If you are celebrating complete 100% success a year from now, what is different about the organization?

If you eliminated your department or failed utterly, what would be different?

Generates Revenue, Sales, Profit

Marketing Public Relations drives sales

Investor Public Relations drives investment

Public Relations drives donations & membership for relevant organizations

Drives Efficiency

Better audience targeting

Reaching more people with a credible message for less money

Avoids Catastrophic Cost

Quality counsel helps to mitigate impacts of crises

How PR Impacts Financial Performance

Increases likelihood to purchase / consider your brand(s)

Minimizes the effects of a crisis

Reinforces communication of organizational values

Rebuilds trust after a crisis

Establishes credibility of new products / companies; ease of market entry

Commanding higher prices, lower costs, premium on stock price

Enhances recommendations / word of mouth leading to faster adoption

Increases customer loyalty / renewals / satisfaction

Improves the attracting / retaining of talent

Lowers legal costs

How PR Positively Affects Reputation / Brand Equity

Increases employee satisfaction and engagement, leading to greater efficiency, increased retention, lower turnover rates, lower recruitment costs, and higher productivity

Lowers legal costs

Change employee behaviors such as greater levels of focus on key areas such as safety, quality, call response times

Provides greater transparency and commitment to and from employees

Creates a platform should it be necessary to communicate bad news at some stage in the future

* Note items here can also refer to other internal publics such as trade association members .

How PR Impacts Employees

Creates public awareness, understanding, and support for legislation, regulation, and political candidates

Affects voter behavior

Helps pass legislation, regulation, and initiatives

Affects specific companies and industries through appropriations, tax impacts, and regulatory changes that can affect any and all aspects of a business

Instigates and perpetuates grassroots or grasstops campaigns

How PR Affects Public Policy

Step 2: Define your investment

You can’t divide by $0

People time

Opportunity Cost

Executive time/goodwill

Step 3: Understand what motivates your stakeholders and priorities the stakeholders

What motivates customers to purchase, members to join, students to apply, etc.

What is your role in the outcome?

Prioritize based on benefits

Step 4: Define your Key Performance Indicators (KPIs)

The Perfect KPI

Gets you where you want to go (achieves corporate goals)

Is actionable

Continuously improves your processes

Is there when you need it

Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measure

Cost savings

Efficiency

Cost per message communicated

Cost per new lead/customer acquired

Productivity:

Increase in employee engagement/morale

Lower turnover/recruitment costs

Engagement:

Ratio of posts to comments

% of repeat visitors

% of 5+min visitors

% of registrations

Trust:

Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)

Thought leadership:

Share of quotes

Share of opportunities

Message penetration

Positioning on key issues

Improvement in favorable/unfavorable ratio

Improvement in Optimal Content Score (OCS)

Step 4: Potential KPIs for Media Relations

Cost Per Message Communicated

Increase in % of discussion containing key messages

% increase in Optimal Content Score

Share of discussion vs. the competition

Share of brand visibility vs. the competition

Share of spokesperson visibility vs. the competition

Share of desirable coverage (positive+neutral coverage) vs. the competition

Share of negatives

Share of favorable positioning on key topics/battles

Share of mentions by key media (including A-list bloggers)

Potential KPIs for thought leader visibility

Our share of quotes in mass media

Our share of papers in scientific/academic/trade journals

Our leaders appointed to key professional boards over time

Our share of papers at key conferences

% awareness among “influentials”

Total opportunities to see our message via speaking engagements

Potential KPIs for Community Relations

Ratio between positive and negative press in local media

% of articles in local media that contain our key messages

% improvement in relationships scores between us and local community and those that influence the local community.

Step 5: Define your benchmarks

Existing benchmarks

Past Performance

Peer companies

Whatever keeps the C-suite up at night

Think 3-5:

A stretch goal

The underdog who’s nipping at your heels

Peer organizations

Anyone that you compete with for share of mind or share of wallet

Past performance: tonality of blog content

The competitive landscape

Non-Profit industry benchmarks in social media % of Desirable Coverage 96% % of Key Message Communication 37% Number of Messages tracked 11.5 Words per Key Message 14 Number of Key Messages tracked 11.5 Most frequent conversation types Express support (69%) Making an observation (28%) % rallying support 3% % asking a question 4% % of exclusive mentions 17% % mentioning brand in title 8% % of discussions in Blogs 51% % of discussion on Twitter 30% % of visibility from YouTube & Flickr 10%

Overview of Key Metrics for Higher Ed Bookmark. Facebook Ext. Blogs Inst. Blogs YouTube MSM SOV 2% — 8% 9% 11% 7% Popularity 230 bkmks 500/mo. — 20 links 150k views — Engagement 59 cmts 1 day 13 cmts 2-12 cmts 2 cmts — % Positive 20% 32% 54% 50% 15% 15% % Negative 0% 0% 4% 0% 1% 2% Strat. Mess. 40% † 18% † 42% 42% † 18% 38%

Step 6: Conduct research (if necessary)

First: find out what already exists

Web traffic

Customer Satisfaction data

Customer Loyalty data

Second: Decide what research is needed to give you the information you need:

Step 6: Selecting a measurement tool Objective KPI Tool Increase inquiries, web traffic, recruitment % increase in traffic #s of clickthrus or downloads Omniture, Google Analytics, Web Trends Increase awareness/preference % of audience preferring your brand to the competition SurveyMonkey, Zoomerang Engage marketplace Conversation index greater than .8 Rankings TypePad, Technorati , Radian6, Communicate messages % of articles containing key messages Total opportunities to see key messages Cost per opportunity to see key messages Media content analysis – Dashboards % aware of or believing in key message Survey

Your tool box needs:

A content source:

Google News/Google Blogs, RSS feeds

Technorati, Social Mention, Twazzup,

Cyberalert, CustomScoop, e-Watch

Radian 6, Techrigy, Sysymos, Visible Technologies, Scout Labs

Survey Monkey/Zoomerang

Your tool box also needs to include:

2. A way to analyze that content

Automated vs. Manual

Census vs random sample

The 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisions

Dashboards to aggregate data

The Content Analysis Process Training Are we all on the same page? Content Collection Did we get everything? Coding Start analyzing (computers/humans) Final Reliability (Resource Permitting) Reporting What does it all mean? Adapted with edits from The Content Analysis Guidebook (Neuendorf , 2002) .

Why an Optimal Content Score?

You decide what’s important:

Benchmark against peers and/or competitors

Track activities against OCS over time

Optimal content score for media coverage

Positive:

Mentions of the brand

Positive brand mentions

Key messages

Customer quoted positive

Analyst quoted

Positioned as trusted partner

Positioned as trustworthy, ethical

Positioned as vendor of choice

Positioned as global

Positioned as one company

Negative

Omitted

Negative tone

No key message

No quote

No analyst quote

How to calculate Optimal Content

Charting OCS over time between divisions

Trend against competition with OCS

Standard classifications of discussion

Acknowledging receipt of information

Advertising something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing agreement

Expressing criticism

Expressing support

Expressing surprise

Giving a heads up

Responding to criticism

Giving a shout-out

Making a joke

Making a suggestion

Making an observation

Offering a greeting

Offering an opinion

Putting out a wanted ad

Rallying support

Recruiting people

Showing dismay

Soliciting comments

Soliciting help

Starting a poll

Validating a position

Standard classifications of videos

Advertisement

Animation

Demonstration

Event/Performance

Fiction

Film

Home Video

Instructional Video

Interview

Lecture

Montage

Music Video

News Broadcast

Promotional Video

Sightseeing/Tour

Slideshow

Speech

Television Show

Video Log

3. A way to measure engagement

The conversation index

Links/trackbacks

Time spent

Relationship studies

Questionnaires

Share of conversation vs share of engagement Page 2 2 1 2 1 6 5 3 1 1 1 1 1 1 1 1 1 1 2 2 2 1 2 1 2 1 4 2 1 4 2 1 1 4 1 6 7 6 2 2 2 2 1 3 2 3 1 0 2 4 6 8 10 12 14 16 18 20 Faculty Students Research, Physical Sciences Courses Research, Earth Sciences Projects, Non - Research Financials Alumni Topics Research, Life Sciences Staff Admissions Legal News Other Research, Agriculture Policies Institution, Overall Campus Life Research, Social Sciences Share of Subject Peer 1 Michigan State Peer 2 Peer 3 Peer 4 15.3% 68.7% 100.0% 4.4% 33.3% 96.8% 28.6% 34.9% 12.5% 43.3% 28.6% 13.0% 38.3% 100.0% 23.6% 66.7% 6.3% 28.6% 20.8% 2.3% 95.6% 33.2% 5.8% 28.6% 100.0% 86.8% 13.0% 31.0% 22.1% 3.2% 71.4% 43.5% 18.8% 94.2% 56.7% 14.2% 13.2% 53.2% 28.4% 21.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Admissions Alumni Topics Campus Life Community Relations Courses Events Faculty Financials Institution, Overall Inventions Legal News Other Partnerships Policies Projects, Non - Research Research, Agriculture Research, Earth Sciences Research, Life Sciences Research, Other Research, Physical Sciences Research, Social Sciences Staff Students Share of Engagement by Subject - ,External Blogs Peer 1 Michigan State Peer 2 Peer 3 Peer 4

The vast majority of discussion in external blogs is neutral. Page

One definition of engagement

% increase or decrease in unique visits

In the past month, what % of all sessions represent more than 5 page views

% of sessions that are greater than 5 minutes in duration

% of visitors that come back for more than 5 sessions

% of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand

% of visitors that become a subscriber

% of visitors that download something from the site

% of visitors that provide an email address

Courtesy of Eric Peterson

5. Measuring ROI

Membership

Registrations

WebSide story/Web trends/Clicktraks

Sales tracking

Marketing Mix Modeling

Research without insight is just trivia

What works, what doesn’t

What needs to be done?

What are you communicating?

What tools work best?

Step 7: Analysis

Data mining the numbers you have

Look for failures first

Then look for exceptional success

Compare to last month, last quarter, last year

Figure out what worked and what didn’t work

Actionable Conclusions A sk for money Get C ommitment Manage T iming I nfluence decisions Get O utside help Just Say N o

Measuring doesn’t have to be complex High Impact Low Cost High Costs Low Impact Blogging Intentional Leaks SEO-optimized press releases e-newsletter Advertising Press conference Party

Take action and measure again

Make sure data is ready when you need it

Work around regular reporting schedules

Keep questions and criteria consistent

Benchmark Conclusions Report

Mistakes made most often in the measurement process

Lack of agreed upon goals

Questionable methodology

Unclear definition of tone/sentiment

Incorrect search strings

Incomplete or out-of-date media list (no CGM)

Incorrect circulation figures

New products/spokespeople not added to analysis parameters

Promising a Jaguar on a bicycle budget

Not allowing enough time to do measurement right

Insufficient so what – looking at the trees not the forest

Thank You!

For more information on measurement, read my blog: http://kdpaine.blogs.com or give me your card and we’ll send you The Measurement Standard, www.themeasurementstandard.com