Social Media Metrics & Measurement WorkshopA SNCR Workshop November 5, 2009Katie Delahaye PaineCEOkdpaine@kdpaine.comwww.measuresofsuccess.comhttp:/kdpaine.blogs.comMember, IPR Measurement Commissionwww.instituteforpr.org

Agenda Basic DefinitionsBasic rules of MeasurementTools, Tips & TechniquesCase StudiesHands on Measurement Workshop

Why Measure? “The main reason to measure objectives is not so much to reward or punishindividual communications manager for success or failure as it is to learn from theresearch whether a program should be continued as is, revised, or dropped in favor of another approach ” James E. Grunig, Professor Emeritus, University of Maryland“If we can put a man in orbit, why can’t we determine the effectiveness of our communications? The reason is simple and perhaps, therefore, a little old-fashioned: people, human beings with a wide range of choice. Unpredictable, cantankerous,capricious, motivated by innumerable conflicting interests, and conflicting desires.”Ralph Delahaye Paine, Publisher, Fortune Magazine , 1960 speech to the Ad Club of St. Louis

First, the numbersYears to Reach 50 million Users: Radio (38 Years), TV (13 Years), Internet (4 Years), iPod (3 Years)…Facebook added 100 million users in less than 9 months…iPhone applications hit 1 billion in 9 months. The fastest growing segment on Facebook is 55-65 year-old females Ashton Kutcher and Ellen Degeneres have more Twitter followers than the entire populations of Ireland, Norway and Panama 80% of Twitter usage is on mobileThere are over 200,000,000 Blogs 54% = Number of bloggers who post content or tweet daily Facebook USERS translated the site from English to Spanish via a Wiki in less than 4 weeks and cost Facebook $0 78% of consumers trust peer recommendations , only 14% trust advertisements Only 18% of traditional TV campaigns generate a positive ROI 90% of people that can TiVo ads do 4

Conquering your fears

A measurement timeline

Old School Metrics AVEsEyeballsHITS (How Idiots Track Success)Couch Potatoes# of Twitter Followers (unless you’re a celebrity)# of Facebook Friends/Fans (unless they donate money) Page 7

Signs that it’s the end of measurement as we know it48% of respondents to a PRWeek study said they were moving $$ out of advertising budgets into Social Media. Only 18% said they were taking $$ away from PR. Procter & Gamble is now paying for engagement, not eyeballs Sodexo cut $300K out of its recruitment budget using TwitterImmunize BC measured SM success via share of discussion, increased awareness and shots given BMC Software measures communications effectiveness based on contribution to EPS HSUS generated $650,000 in new donations from an on-line photo contest on FlickrThe Red Cross measures the effectiveness of Twitter via lives saved and property lostIBM predicts the ends of advertising as we know. Also receives more leads, sales and exposure from a $500 podcast than it does from an ad11 Mom’s turned around Wal-Mart's image and delivered measureable increases in sales.

Page 9You are a party planner, not a communicator 21st Century Old School

Page 10Social Media renders everything you know about measurement obsoleteOld School PR21st Century Role of PR The definition of timely has changedThe definition of reach has changedGRPs & Impressions are impossible to count (an irrevelvant) in social media The definition of success has changedThe answer isn’t how many you’ve reached, but how those you’ve reached have responded

The Engagement Decision Tree

Goals for Social Media Marketing/leads/sales/Mission/safety/civic engagementRelationship/reputation/positioning To fix thisOr get to this

Goals drive metrics, metrics drive results 13GoalMetrics

Change the conversation, improve your reputation Improve your reputationListen first, then respondStop doing stupid things

Negative coverage over time

Correlation exists between traffic to the ASPCA web site and the organization’s overall media exposure

Tying activity to development/marketing goals17

What do you need to measure?

Goals, Actions and Metrics

The 7 steps to Social Media ROI Define the “R” – Define the expected results?Define the “I” -- What’s the investment?Understand your audiences and what motivates them Define the metrics (what you want to become) Determine what you are benchmarking againstPick a tool and undertake researchAnalyze results and glean insight, take action, measure again

Step 1: Define the “R” Why Social Media? What return is expected? – Define in terms of the business or mission.What problems is Social Media supposed to solve? What were you hired to do? What difference are you expected to make? If you are celebrating complete 100% success a year from now, what is different about the organization?If your Social Media is eliminated, what would be different? 21

Step 2: Define the “I” What is the investment? PersonnelAgency compensationSenior Staff time Opportunity cost22

Step 3: Define your audiences and how you impact themThere is no “audience.” There are multiple constituencies Should you blog or Twitter? Don’t ask me, ask your customers List every stakeholderWhere do they go for information?What’s important to them?What is the benefit of having a good relationship with that stakeholder group?What’s important to them?Where do they go for information?What do you want them to know? Understand your role in getting the audience to do what you want it to doRaise awarenessIncrease preferenceIncrease engagement23

Step 4: Define your Key Performance Indicators (KPIs)24The Perfect KPIGets you where you want to go (achieves corporate goals)Is actionableContinuously improves your processesIs there when you need itKPIs should be developed for: Your own propertiesDifferent tacticsOther influential sites

Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measureCost savingsEfficiencyCost per message communicatedCost per new lead/customer acquiredProductivity: Increase in employee engagement/moraleLower turnover/recruitment costsEngagement: Ratio of posts to comments% of repeat visitors% of 5+min visitors% of registrationsTrust:Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)Thought leadership: Share of quotesShare of opportunities Message penetrationPositioning on key issuesImprovement in favorable/unfavorable ratioImprovement in Optimal Content Score (OCS)25

KPIs for External blogs and other Consumer Generated Media Share of positioningShare of rants vs. ravesShare of positives/negativesShare of visibilityShare of quotesShare of brand benefits mentionedTypes of conversationsEngagement – ratio of posts to comments Optimal content score

Revenue KPIs Cost savingsCost per click thru, downloads, engagement vs other marketing channelsCost per message communicated vs other channelsLifetime value of engagementCost per customer acquisition

Engagement metrics % increase or decrease in unique visits In the past month, what % of all sessions represent more than 5 page views % of sessions that are greater than 5 minutes in duration % of visitors that come back for more than 5 sessions % of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand % of visitors that become a subscriber % of visitors that download something from the site % of visitors that provide an email addressRatio of posts to comments Courtesy of Eric Peterson

Emerging benchmarks Past PerformanceThink 3PeerUnderdog nipping at your heelsStretch goal Whatever keeps the C-suite up at nightStep 5: Define your benchmarks29

Emerging benchmarks Past PerformanceThink 3PeerUnderdog nipping at your heelsStretch goal Whatever keeps the C-suite up at nightStep 5: Define your benchmarks30

Past performance: tonality of blog content

The competitive landscape

Blogs and Twitter dominated the social media landscape33

Consumer organizations are far more likely to see Undesirable Discussion than non-profits or educational institutions. 34*For each mention, we determine whether it leaves the reader more or less likely to donate to, partner with, volunteer for or otherwise support the efforts of Non-Profit Organization. If it leaves the reader more likely, we consider it positive. If it leaves the reader less likely, we consider it negative. If it doesn’t sway the reader one way or the other, we consider it neutral.

Key Message penetration lags the non-profit average 35

Focus on fewer key messages with shorter statements36Recommendation: Less is more when trying to get your messages across. We recommend reducing the number of key messages and simplifying and shortening the key messages that are tracked.

Most conversations were making observations rather than expressing support37

Consumer organizations tend to be the focal point of more conversations38

Consumer companies saw a significantly higher level of visibility than non-profits39

Overview of Key MetricsPeer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.Actions attributed to individuals were responsible for most content, except on YouTube.

Few subjects appear across all forms of social media, so tailor outreach accordingly

Step 6: Pick a tool Content AnalysisSurveyWeb Analytics

Step 6: Selecting a measurement tool43

Content Analysis requires a content source: Free: Google News/Google Blogs, RSS feeds, Technorati, Social Mention, Twazzup, $500+Radian 6, Techrigy, Sysymos, Visible Technologies, Scout Labs, Cyberalert, CustomScoop, e-Watch44

A way to analyze content AutomatedHuman: Census vs random sampleSentiment vs TopicsThe 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisions45

A coding methodologyTonalityWhat messages were communicatedHow you’re positioned on key issuesDominance/Prominence/VisibilityAuthoritySubject of the article/postingWho was quoted?Products, events, initiatives, battles mentionedOptimal Content Score

What matters, what doesn’t

Standard classifications of discussionResponding to criticismGiving a shout-outMaking a jokeMaking a suggestionMaking an observationOffering a greetingOffering an opinionPutting out a wanted adRallying supportRecruiting peopleShowing dismaySoliciting commentsSoliciting helpStarting a pollValidating a positionAcknowledging receipt of informationAdvertising somethingAnswering a questionAsking a questionAugmenting a previous postCalling for actionDisclosing personal informationDistributing mediaExpressing agreementExpressing criticismExpressing supportExpressing surpriseGiving a heads up

Responding to criticism

Giving a shout-out

Making a joke

Making a suggestion

Making an observation

Offering a greeting

Offering an opinion

Putting out a wanted ad

Rallying support

Recruiting people

Showing dismay

Soliciting comments

Soliciting help

Starting a poll

Validating a position

Acknowledging receipt of information

Advertising something

Answering a question

Asking a question

Augmenting a previous post

Calling for action

Disclosing personal information

Distributing media

Expressing agreement

Expressing criticism

Expressing support

Expressing surprise

Giving a heads up

Standard classifications of videosAdvertisementAnimationDemonstrationEvent/PerformanceFictionFilmHome VideoInstructional VideoInterviewLectureMontageMusic VideoNews BroadcastPromotional VideoSightseeing/TourSlideshowSpeechTelevision ShowVideo Log

Why an Optimal Content Score? You decide what’s important:Benchmark against peers and/or competitorsTrack activities against OCS over time Positive: Mentions of the brandKey messagesPositioningVisibilityNegative OmittedNegative toneNo key message50

How to calculate Optimal Content

Charting OCS over time between divisions

Surveys require: A defined sampleA list – a way to get to that sampleAgreement on what questions you need to answerA survey instrument/questionnaire A testA way to analyze dataSPSSSAS53

Aspects of relationships Control mutualityTrustSatisfactionCommitmentExchange relationshipCommunal relationship54

Components of a Relationship IndexControl mutuality In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)This organization really listens to what people like me have to say.TrustThis organization can be relied on to keep its promises.This organization has the ability to accomplish what it says it will do.SatisfactionGenerally speaking, I am pleased with the relationship this organization has established with people like me.Most people enjoy dealing with this organization.CommitmentThere is a long-lasting bond between this organization and people like me.Compared to other organizations, I value my relationship with this organization moreExchange relationshipEven though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.This organization will compromise with people like me when it knows that it will gain something.This organization takes care of people who are likely to reward the organization.Communal relationshipThis organization is very concerned about the welfare of people like me.I I think that this organization succeeds by stepping on other people. (Reversed)

How to implement relationship metrics Step 1: Conduct a benchmark relationship studyStep 2: Implement PR programStep 3: Conduct a follow up relationship studyStep 4: Look at what’s changed

Web Analytics Require: Google Analytics/Web Trends/OmnitureUnique URLsData delivered in parallel with content analysisAbility to correlate and integrate data SPSS/SAS57

Step 7: Analysis - -Research without insight is just triviaLook for failures firstCheck to see what the competition is doing Then look for exceptional successCompare to last month, last quarter, 13-month averageFigure out what worked and what didn’t workMove resources from what isn’t working to what is 58

Ask for money Get Commitment Manage TimingInfluence decisions Get Outside help Just Say NoActionable Conclusions59

Best Practices:Correlations to bottom-line impactDonationsMembershipsSign-upsLeadsUsing SMM for planningDefine the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversationBenchmarking against your peersLooking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeopleSocial Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product Improve your reputationListen first, then respondStop doing stupid things

Using SMM for planningThe environmental scanDefining issues in a marketSelecting a positioning that works

Case Studies

For all institutions, most postings were simply making an observation or distributing media. Page 63cx

Share of conversation vs share of engagement Page 64Share of Engagement by Subject -,External BlogsShare of SubjectStudents23.6%33.2%22.1%21.1%Staff100.0%Research, Social Sciences141Research, Social Sciences4.4%95.6%Campus LifeResearch, Physical Sciences138.3%2.3%31.0%28.4%Research, OtherInstitution, Overall213Research, Life Sciences13.0%20.8%13.0%53.2%Policies2Research, Earth Sciences86.8%13.2%Research, Agriculture4Research, Agriculture100.0%Projects, Non-ResearchOther28.6%28.6%28.6%14.2%1Policies100.0%Legal NewsPeer 112PartnershipsMichigan StateAdmissions11Peer 1OtherPeer 2StaffMichigan State1Legal News43.3%56.7%Peer 3InventionsPeer 2Research, Life Sciences11213Peer 4Institution, Overall5.8%94.2%Peer 3Alumni Topics11Financials68.7%12.5%18.8%Peer 4Financials212Faculty15.3%34.9%6.3%43.5%Projects, Non-ResearchEvents1112Courses28.6%71.4%Research, Earth Sciences122Community RelationsCourses12Campus LifeResearch, Physical Sciences3246Alumni Topics96.8%3.2%AdmissionsStudents33.3%66.7%5217Faculty262260%10%20%30%40%50%60%70%80%90%100%02468101214161820

The vast majority of discussion in external blogs is neutral.Page 65

Case Study: Establishing benchmarks at Georgia Tech

Quantity and quality of discussion of Georgia Tech and four peer institutions across relevant user-generated media (UGM) channels in order to:Establish performance benchmarksObserve user habits to inform UGM strategiesUnderstand the influence of traditional media on UGM channelsProvide support for funding of UGM programsCase Study: Georgia Tech

Establish performance benchmarks

Observe user habits to inform UGM strategies

Understand the influence of traditional media on UGM channels

Provide support for funding of UGM programs

Overall Comparison of Georgia Tech Social Media Outlets68/17Based on 2007 data, Georgia Tech outperformed its peers in Facebook presence, but significantly lagged peers on other social media. Post-2007 media monitoring has not included a social media dimension due to funding constraints, but this will be important to trend as feasible in the future.Share of All CoverageDefinitions: YouTube: a video sharing site. Social Bookmarking: a site where members can display media they have found on the web. Facebook: a social networking site. Institutional Blogs: blogs hosted and owned by schools studied. External Blog: any blog post that is not hosted by an institution.

Based on 2007 data, Georgia Tech outperformed its peers in Facebook presence, but significantly lagged peers on other social media.

Post-2007 media monitoring has not included a social media dimension due to funding constraints, but this will be important to trend as feasible in the future.

User Generated Media69During a crisis, UGM channels more likely to be negativeUGM amplified negative traditional media coverage.Unusual negative stories, like MIT’s fake bomb scare, became popular on social bookmarking sites. Negative news linked to politics was a mainstay on external blogs.Facebook profiles amplified each of these effects, and also included critical pieces from campus newspapers.

70UGM channels offered equal opportunity for message communication

Peer 1 dominates social bookmarking, institutional blogs, Peer 4 leads on Facebook

Where people get the content they share on FacebookSources of content Genre of content

Influence of traditional media On average, bloggers included as many as six links to external content in a post, the number three source being traditional news media sites.Links to its newsroom accounted for 26% of links to mit.edu on blogs.On Facebook, traditional news media sites were the source of 25% of popular items posted to profiles.One third of content on social news sites was from traditional media sources.Twice as many hard news stories were posted to social news sites as features.Selected Traditional Media Outlets Among Popular Sources of ContentBBC Boston Globe CNET CNNEurekAlert! Google News Los Angeles Times The New York TimesPittsburgh Post-Gazette San Francisco Chronicle Washington Post

Focus on Social BookmarkingIn the event of a crisis, expect seeding from local papersThursday & Friday saw the greatest number of seeds. GIT’s status as a technical institution is an asset in the social bookmarking environment Few strategic messages appeared in social bookmarking sites

USO Case Study

76

Moving conversation from observation to support

Social Media OverviewMarch 200978

Media Engagement & Online Giving35,152,789 OTSRed line indicates media impressions6,253,852 OTS

Best Practices:Correlations to bottom-line impactDonationsMembershipsSign-upsLeadsUsing SMM for planningDefine the time frame, market/topic you want to studyUse Google News, Technorati or Radian6 to identify the conversations around the topic Analyze the conversations for type, tone and positioningLook at share of positioning, tone or conversationBenchmarking against your peersLooking at what the best doSetting goals accordinglyUse data to persuade recalcitrant spokespeopleSocial Media in CrisisListen instantly to a wide range of influencersIdentify weaknesses in communications, customer service, or in the product Improve your reputationListen first, then respondStop doing stupid things

Thank You!For more information on measurement, read my blog: http://kdpaine.blogs.com or subscribe to The Measurement Standard: www.themeasurementstandard.comFor a copy of this presentation go to: http://www.kdpaine.comFollow me on Twitter: KDPaineFriend me on Facebook: Katie Paine Or call me at 1-603-868-1550