Build Your Bottom Line With Data-Driven Public Relations: Advanced ROI, KPI and ROMI Measurement February 13, 2009Katie Delahaye PaineMember, IPR Measurement Commissionwww.instituteforpr.comCEOkdpaine@kdpaine.comwww.kdpai.comkdpaine.blogs.com

Why Measure? “The main reason to measure objectives is not so much to reward or punishindividual communications manager for success or failure as it is to learn from theresearch whether a program should be continued as is, revised, or dropped in favor of another approach ” James E. Grunig, Professor Emeritus, University of Maryland“If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

AgendaGeneral Principles of MeasurementThe immutable Laws of 21st Century Communications7 Steps to Building the perfect measurement systemTools You’ll Need Design your own measurement workshop

Conquering your fears

A measurement timeline

Social Media renders everything you know about measurement obsoleteOld School PR21st Century Role of PR You can’t talk when the marketing department is talking sales, leads and revenueImpressions are impossible to count in social media Who cares about impressions when you can measure brand engagement ?Who cares about reach when you can measure revenue? Wouldn’t you rather be closer to your customers? Better measurement tools are available Page 6

Why Traditional Metrics (AVEs) are like buggy whips They both confuse activity with outcomeThe goal is to arrive at your destination, not to have a faster horseThey both were based on a flawed premiseNeither PR people nor horses respond well to beatingsComparing PR to Advertising is like comparing the Surrey With a Fringe On Top to the Space Shuttle – they serve different purposesThey both were rendered obsolete by technology Page 7

12 Signs that it’s the end of measurement as we know itIn Iran, people armed with Twitter and cell/phones are fighting guys with guns and possibly winning. The State Dept, and the Dept of Defense considers Twittering and other forms of social media critical to national securityBMC Software measures communications effectiveness based on contribution to EPS BestBuy measures 85% lower turnover as a result of its Blue Shirt communityState Farm uses an internal blog to measurably improve moraleASPCA correlates increases in on-line donations and increased membership with its social media effortsHSUS generated $650,000 in new donations from an on-line photo contest on FlickrNWF increased wildlife spotting as well as members with its Twitter account The Red Cross measures the effectiveness of Twitter via lives saved and harm avoidedImmunizeBC measures success in terms of vaccines given, awareness AND traffic IBM receives more leads, sales and exposure from a $500 podcast than it does from an ad11 Mom’s turned around Walmart’s image and delivered measureable increases in sales. @comcastcares turned around Comcast’s customer service reputation A social media campaign for War Child delivered a 38% increase in donations and 300 new volunteers

The Engagement Decision Tree9

The immutable laws of 21st Century PR Measurement There is no market for your messageAll the benchmarks have changedSize doesn’t matter so stop screaming, start listeningIt’s not how many eyeballs, it’s the right eyeballsHITS = How Idiots Track SuccessROI doesn’t mean what you think it does because you can’t divide by zeroYou become what you measure, so match the measurement tool to your objective10

The measurement fork in the road Marketing/leads/salesReputation/relationshipsTo fix thisOr get to this

Goals drive metrics, metrics drive results GoalMetrics12

Changing reputation with charts Tone of Conversation over time 6050304023016MentionsPositiveNeutral17Negative5201227244212029310161588105994625444422222220DecJanFebMarAprMayJunJulAugSepOctNovDec20072008

Negative coverage over time

Correlation exists between traffic to the ASPCA web site and the organization’s overall media exposure

Tying activity to development/marketing goals16

CEOS think its important, Marketers don’t CEOs are more likely than Marketers to value PR highlyDominant coalitions tend to value and support communicators who first demonstrate their worth When PR reports to the C-Suite they measure:Crisis avoidance/mitigation Influence on corporate culture Influence on corporate reputation Influence on employee attitudes Influence on stakeholder awareness Influence on stakeholder opinions When PR reports to marketing they measure :Contribution to sales Total circulation of clips Total number of clips Number of clips in “top tier” media(Source: 2005 PR GAP IV Study, Strategic Public Relations Center, USC Annenberg School of Communications)17

How PR fits into ROMI PR delivers measurable impact on:PositioningAwarenessPreferenceSales/revenueMarket shareWeb trafficCost savings When factored in to marketing mix modeling, PR delivers significantly higher value 18

PR delivers more results for less money Miller discovered that PR campaigns generate 4% of incremental sales compared to 17.3% of incremental sales for TV.However, PR delivered that 4% for less than 1% of the budget. 19

PR is shown to deliver more value A major consumer company found that PR delivered 8 times the value of TV and 4 times the value of trade advertising. 20

Proof of PR’s impact on sales P&G found that PR drives salesThree of the six products showed PR with the highest ROI of any marketing tactic Overall PR delivered a 275% ROIAT&T found that PR delivered customers at a fraction of the cost 21

What do you need to measure?

Goals, Actions and Metrics 23

The 7 steps to PR ROI Define the “R” – Define the expected results?Define the “I” -- What’s the investment?Understand your audiences and what motivates them Define the metrics (what you want to become) Determine what you are benchmarking againstPick a tool and undertake researchAnalyze results and glean insight, take action, measure again24

Step 1: Define the “R” What return is expected? What were you hired to do? If you are celebrating complete 100% success a year from now, what is different about the organization?If your department was eliminated, what would be different? 25

Step 2: Define the “I” What is the investment? PersonnelAgency compensationSenior Staff time Opportunity cost26

Step 3: Define your audiences and how you impact themThere is no “audience.” There are multiple constituencies List every stakeholderWhere do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?Understand your role in getting the audience to do what you want it to doRaise awarenessIncrease preferenceIncrease engagement27

Understand your environment: The 7WsWho are the audiences? (Demographics & Psychographics)What issues are important to them?What are they seeing now?Where do they go for information?What do they think about you now?What do you want them to do/think/say/write?What do you need to do about it?

What makes a perfect communications KPI?Gets you where you want to go (achieves corporate goals) Is actionable by individuals as well as departmentsContinuously improves your processes Is there when you need it

Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measureCost savingsEfficiencyCost per message communicatedCost per new lead/customer acquiredProductivity: Increase in employee engagement/moraleLower turnover/recruitment costsEngagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrationsTrust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)Thought leadership: Share of quotesShare of opportunities Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)30

Potential Metrics for Media Cost Per Message CommunicatedShare of discussion vs. the competitionShare of brand visibility vs. the competitionShare of recommendations (positive coverage) vs. the competitionShare of spokesperson visibility vs. the competitionShare of negativesShare of coverage of key topicsShare of rants and raves in the blogosphere Share of mentions by key media 31

Measuring the impact of messagingPercent of impressions containing messages by productA major software company had had many different product launches over the course of a year. By measuring the extent to which key messages were communicated, versus those messages they didn’t want to see in print they were able to judge the effectiveness of each launch. Further, by investigating the large number of negative messages they were able to isolate problems with a specific group of media. Turns out, the person responsible for that group of publications was sidetracked by her wedding and hadn’t returned phone calls. 32

Metric: Cost per message communicated The press tour was clearly the most efficient for communicating key messages and the big party was least efficient. Measuring which tactic was most efficient33

Potential metrics for blogsNumber of unique usersReturning versus new readersReferring source statisticsLinks from other sitesRankConversation Index: The ratio of blog comments to blog posts (where applicable)Total time spent on the siteThe popularity of the content itself, which gets the most views Traffic from blog to web siteSales34

Potential Metrics for Community RelationsRatio between positive and negative press in local media % of articles in local media that contain our key messages% improvement in relationships scores between us and local community and those that influence the local communityLower litigation costs 35

Why an Optimal Content Score? You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive: Mentions of the brandKey messagesPositioningVisibilityNegative OmittedNegative toneNo key message36

How to calculate Optimal Content

Standard classifications of discussionResponding to criticismGiving a shout-outMaking a jokeMaking a suggestionMaking an observationOffering a greetingOffering an opinionPutting out a wanted adRallying supportRecruiting peopleShowing dismaySoliciting commentsSoliciting helpStarting a pollValidating a positionAcknowledging receipt of informationAdvertising somethingAnswering a questionAsking a questionAugmenting a previous postCalling for actionDisclosing personal informationDistributing mediaExpressing agreementExpressing criticismExpressing supportExpressing surpriseGiving a heads up

Responding to criticism

Giving a shout-out

Making a joke

Making a suggestion

Making an observation

Offering a greeting

Offering an opinion

Putting out a wanted ad

Rallying support

Recruiting people

Showing dismay

Soliciting comments

Soliciting help

Starting a poll

Validating a position

Acknowledging receipt of information

Advertising something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing agreement

Expressing criticism

Expressing support

Expressing surprise

Giving a heads up

For all institutions, most postings were simply making an observation or distributing media. cxPage 39

Past PerformanceThink 3PeerUnderdog nipping at your heelsStretch goal Whatever keeps the C-suite up at nightStep 5: Define your benchmarks40

Overview of Key MetricsPeer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.† Small base size. Findings are directional only.

Internet dominated in 5 out of 6 months Page 42

Stanford dominates in broadcast coveragePage 43

Benchmarks put numbers in perspectivePage 44

MIT Berkeley see highest visibility in Social MediaPage 45

First: find out what already existsWeb analyticsCustomer Satisfaction dataCustomer loyalty dataSecond: Decide what research is needed to give you the information you need: Message content analysisRelationship surveysStep 6: Conduct research (if necessary)46

Step 6: Selecting a measurement tool47

The Basics48Types of measuresSurvey ResearchContent analysis Web AnalyticsUnique visitorsSubscribersCost per click thruCost per contact# of downloads Conversation indexRankings

Your tool box needs: A content source: Google News/Google BlogsTechnorati, Social Mention, Twazzup, Cyberalert, CustomScoop, e-WatchRadian 6, Techrigy, Visible Technologies, Crimson Hexagon, Sysymos, Umbria RSS feedsSurvey Monkey/Zoomerang49

Your tool box also needs to include: 2. A way to analyze that contentAutomated vs. Manual Census vs random sampleThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisionsDashboards to aggregate data Tools:WoopraNet promoter scoreHubspot GraderXinureturnsTwinfluenceSPSSExcelCrimson Hexagonwww.tealium.com50

Woopra

Net promoter score

Hubspot Grader

Xinureturns

Twinfluence

SPSS

Excel

Crimson Hexagon

www.tealium.com

Your tool box also needs to include: 3. A way to measure engagementThe conversation index=Ratio of posts to comments Relationship studiesThe engagement index51

Ratio of posts to comments

Managing engagement on your own property% increase or decrease in unique visits In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email addressCourtesy of Eric Peterson

Share of conversation vs share of engagement Share of Engagement by Subject -,External BlogsShare of SubjectStudents23.6%33.2%22.1%21.1%Staff100.0%Research, Social Sciences141Research, Social Sciences4.4%95.6%Campus LifeResearch, Physical Sciences138.3%2.3%31.0%28.4%Research, OtherInstitution, Overall213Research, Life Sciences13.0%20.8%13.0%53.2%Policies2Research, Earth Sciences86.8%13.2%Research, Agriculture4Research, Agriculture100.0%Projects, Non-ResearchOther28.6%28.6%28.6%14.2%1Policies100.0%Legal NewsPeer 112PartnershipsMichigan StateAdmissions11Peer 1OtherPeer 2StaffMichigan State1Legal News43.3%56.7%Peer 3InventionsPeer 2Research, Life Sciences11213Peer 4Institution, Overall5.8%94.2%Peer 3Alumni Topics11Financials68.7%12.5%18.8%Peer 4Financials212Faculty15.3%34.9%6.3%43.5%Projects, Non-ResearchEvents1112Courses28.6%71.4%Research, Earth Sciences122Community RelationsCourses12Campus LifeResearch, Physical Sciences3246Alumni Topics96.8%3.2%AdmissionsStudents33.3%66.7%5217Faculty262260%10%20%30%40%50%60%70%80%90%100%02468101214161820Page 53

The vast majority of discussion in external blogs is neutral.Page 54

Developing your coding framework: What do you need to know? TonalityWhat messages were communicatedHow you’re positioned on key issuesDominance/Prominence/VisibilitySubject of the article/postingWho was quoted?

Popular Concepts in Media AnalysisTonalityMessages communicatedPositioning on key issuesDominance/Prominence/VisibilityPrimary SubjectQuoted personsProducts, events, initiatives mentioned

Building MeasuresUnits of contentOverall theme (when many messages are combined)Entire message (e.g. article, blog post, etc.)Message parts (brand mentions, paragraphs, sentences)Types of contentManifest: on the surfaceLatent: the meaning or interpretation of the contentLatent Pattern: meaning determined by surface observationsLatent Projective: meaning determined by coder interpretation

Building Measures (cont.)Variables are exhaustive/ mutually exclusiveOne can always be selected, and only oneTypes of variablesNominal: categories, “buckets”Brands mentioned, organizations mentioned, messages communicatedOrdinal: categories with an order or scaleTonality, prominence, dominanceNumbers: number of words (zero means no words), number of brand mentions

Testing the Coding FrameworkTests of reliabilityNominal, ordinal (0.00 – 1.00)Crude agreementScott’s piCohen’s kappaKrippendorf’s alphaInterpreting Reliability≥ 0.90 Excellent!≥ 0.80 Good!≥ 0.67 Be careful!≤ 0.67 Refine your instrument

ROI measurement tools Google AnalyticsMembership/donations/signupsIntent to purchase RegistrationsMoney savedMarketing Mix Modeling60

To do Marketing Mix Modeling you MUST track TonalityProminence/visibilityDominanceShare of discussionShare of favorable/unfavorable positioning on key issues that drive customer behaviorShare of recommendationsShare of brand benefitsShare of brand mentions61SalesGRPsContent Analysis Web data Media Demographic analysis

Ways to quantify the data Eyeballs – NOT$$ BudgetsFriends/followersPage views, Web analyticsPanelsSurveys62

A Proposed Engagement Index Output OuttakeOutcomeClickthruDonations/ordersSignupsTime on siteRepeat visits Forwards/links /commentsRelationshipsTone/content of conversationMembership++An engagement index?

11 numbers your web analytics guru should give you every month% increase or decrease in unique visits Change in page rank - i.e. a list of the top ten most popular areas and how it has changed in the last week How many sessions on our blog or web site represent more than 5 page views In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email addressCourtesy of Eric Peterson

Relationships = Reputation and brand You can manage relationships, you can’t “manage” your reputationBrands are dead, long live LovemarksIt’s all about stories, conversations, and the experience You can’t measure stories, conversations or the experience without measuring relationships 65

Aspects of relationships Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship66

Control MutualityThe degree to which parties agree on who has the rightful power to influence one another. Although some imbalance is natural, stable relationships require that organizations and publics each have some control over the other.67

Questions that test Control MutualityThis organization and people like me are attentive to what each other says.This organization believes the opinions of people like me are legitimate.In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.The management of this organization gives people like me enough say in the decision-making process.68

Measuring Trust One party’s level of confidence in and willingness to open oneself to the other party. Includes: Integrity: the belief that an organization is fair and justDependability: the belief that an organization will do what it says it will doCompetence: the belief that an organization has the ability to do what it says it will do.69

Questions to measure trustThis organization treats people like me fairly and justly.Whenever this organization makes an important decision, I know it will be concerned about people like me.This organization can be relied upon to keep its promises.I believe that this organization takes the opinions of people like me into account when making decisions.I feel very confident about this organization’s skills.This organization has the ability to accomplish what it says it will do.70

Measuring satisfactionThe extent to which each party feels favorably toward the other because positive expectations about the relationship are reinforced. A satisfying relationship is one in which the benefits outweigh the costs.71

SatisfactionI am happy with this organization.Both the organization and people like me benefit from the relationship.Most people like me are happy in their interactions with this organization.Generally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.72

Measuring commitment The extent to which each party believes and feels that the relationship is worth spending energy to maintain and promote. 73

CommitmentI feel that this organization is trying to maintain a long-term commitment to people like me.I can see that this organization wants to maintain a relationship with people like me.There is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization more.I would rather work together with this organization than not.74

Measuring relationshipsExchange Relationship In an exchange relationship, one party gives benefits to the other only because the other has provided benefits in the past or is expected to do so in the future.Communal Relationship In a communal relationship, both parties provide benefits to the other because they are concerned for the welfare of the other -- even when they get nothing in return.75

Exchange RelationshipsWhenever this organization gives or offers something to people like me, it generally expects something in return.Even though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.76

Communal RelationshipsThis organization does not especially enjoy giving others aid. (Reversed)This organization is very concerned about the welfare of people like me.I feel that this organization takes advantage of people who are vulnerable. (Reversed)I think that this organization succeeds by stepping on other people. (Reversed)This organization helps people like me without expecting anything in return.77

Relationship survey techniques78

Putting your tool to work Define your instrument QuestionnaireReader instructionsTest your instrumentAre options mutually exclusive? 79

What to look for in a research vendor AccuracyAdherence to the IPR and other industry standard practicesSolid research designSolid samplingQuality control mechanismsProven track record in the selected methodology Transparency of methodology 80

Step 7: AnalysisResearch without insight is just triviaWhat works? What doesn’t?What needs to be done? What are you communicating?What tools work best?81

What works? What doesn’t?

What needs to be done?

What are you communicating?

What tools work best?

Data mining the numbers you have Look for failures firstThen look for exceptional successCompare to last month, last quarter, last yearFigure out what worked and what didn’t work 82

Ask for money Get Commitment Manage TimingInfluence decisions Get Outside help Just Say NoActionable Conclusions83

Factors in budgeting Acquiring content can be third of your costNumber of competitors trackedNumber of publications/media outletsSize of universe being measuredLength of survey Type of analysis: automated vs. human Type of reporting: automated, vs. written; insight and conclusions vs. simple reporting Frequency of reporting 84

Budgeting for PR measurement Content30% of the costMust gather consistently for client and competitorsNeed at least two years worth of data $3 a clip -- $300/monthAnalysisDo-it-yourself $500 a monthAutomated analysis: $20K - $500KManual: $5K-$500KMarketing Mix Modeling: $100K-$500K+85

Ways to make the budget fit the task Reduce the volumeSampleBecome a research project for your local university 86

8 ways to do research without a budgetBecome someone’s research projectInvolve your board of directors and volunteersResearch something that HAS a budgetTake advantage of free offersBecome a case studyTeam up with peer organizationsAnalyze data that already existsUse blogs and social networks to listen to conversations 87

A dozen mistakes made most often in the measurement process12: Mismatched goals and metrics11: Questionable analysis methodology10: Unclear definition of tone9: Incorrect search strings8: Incomplete or out-of-date media list7: Lack of clarity on geography 6: Incorrect circulation figures5: Incomplete or out-of-date messages, spokespeople and/or products4: Not collecting sufficient data to draw conclusions 3: Promising a Jaguar on a Segway budget2: Not allowing enough time to do measurement right1: Insufficient so what? – looking at the trees not the forest88

Take action and measure againMake sure data is ready when you need itWork around regular reporting schedulesKeep questions and criteria consistent89

Share of exposure vs. the competition over time100%90%80%Intel70%60%TI50%Moto40%30%National20%10%0%JulJulJanJanJanSepSepMarMarNovNovMayMayLesson learned, you need the PR department 90

Trade show/eventIndustry issueProduct reviewRelease + conferenceNo MessagePress release plus VNRNegative MessagePositive MessageMedia advisoryExec InterviewContract winsApplication articles0%20%40%60%80%100%91Interviews and media advisories generated best coverage

Facebook: Correlating MSM, CGM and signups Strong correlationNon-negative discussion only92

Comparing the effectiveness of different tactics93

Measurement helps improve message communication94

Measurement improves spokesperson visibility95

Case Study: Putting a value on the Primary NH’s First in the Nation Primary attracts nearly 4000 journalists to the stateNH’s image in the past has been: “persnickety,” “flannel shirted yokels,” “worst economy in the country,” “last without Martin Luther King Day,” 96

Specific endorsements of NH increased97

Negatives were reduced, positives increased98

Key Messages Increased as Well 99

Case study NH: Tying the primary to tourism value$33 millionin potential tourismrevenue660,000 non-primary visits(3% of 22 million)22 million positive impressions (10% positive impressions) 220 million media impressions100

The impact of media coverage on business development660,000 visits to NH (3% of 22 million)132 new businesseswith avg. 20 emp. per business = 2,640 employees@ $10,000 subsidy/business development value per emp.= $26,400,000 13,200 corporate decision- makers and entrep. (2% of 660,000 - actual US figure is 11%) 132 new businesses(1% of 13,200 or .0002 of all visits)101

Tying sales to PR Porter Novelli had found a correlation between PR share of discussion and prescription sales of OxytrolThe more messages the media coverage contained, the greater the level of salesSpecific messages yielded greater salesSource: IPR Jack Felton Golden Ruler of Measurement Award 2006 www.instituteforpr.org102

Thank You!For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard: www.themeasurementstandard.comFor a copy of this presentation go to: http://www.kdpaine.comOr call me at 1-603-868-1550Or email me at kdpaine@kdpaine.com103