Measuring Social Media Prsa Digital Impact


Published on

My preso for PRSA\'s Digital Impact Conference, NYC, May 5, 2011.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Are they an idea starter or spreader; source or spider?  They may be prolific but are they prominent?  Are they highly inter-related, inter-connected, and centrally located in the network?  How engaged is this person’s following in a dialogue?  Topicality Here we need to recognize, though, that authority is contextual and topical.  One might be an authority on PR measurement but not on 18th century Russian literature. 
  • Measuring Social Media Prsa Digital Impact

    1. 1. 7 Cs of MEASURING SOCIAL MEDIA PRSA DIGITAL IMPACT CONFERENCE MAY 6, 2011 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 Alan Chumley Senior Vice President @alanchumley / @CARMA_Tweets
    2. 2. ALAN CHUMLEY, SVP <ul><li>14 years in the industry </li></ul><ul><li>Recovering PR practitioner: ING, Bell Canada </li></ul><ul><li>Former Director of Measurement @ Hill & Knowlton </li></ul><ul><li>Masters in Communications & Culture: </li></ul><ul><ul><li>Research foci: media effects & uses, reception analysis, audience studies </li></ul></ul><ul><li>University-level adjunct instructor of PR measurement </li></ul><ul><li>LinkedIn early 2006 </li></ul><ul><li>Blogging / Tweeting since early 2007 </li></ul>
    3. 3. ISN’T IS <ul><li>An emerging / experimental philosophy / approach / filter / lens through which to look at measuring social media </li></ul><ul><li>A high level connected / systems way of approaching the problem </li></ul><ul><li>About communities not an audience </li></ul><ul><li>More sociologist than publicist </li></ul><ul><li>Not a Case Study </li></ul><ul><li>Not a mention of Old Spice in sight </li></ul><ul><li>Not a specific ‘how to’ / no one-size-fits all </li></ul><ul><li>Not focused on widgets or mobile </li></ul><ul><li>Doesn’t focus on a singular or few social media platforms—looks at all holistically </li></ul>@alanchumley / @CARMA_Tweets
    4. 4. MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES Source: Harvard Business Review: the New Conversation: Taking Social Media from Talk to Action Surveyed 2100 Harvard Biz Review subscribers @alanchumley / @CARMA_Tweets Why is this the case? Gap between leap of faith true believers and C-suite’s need for metrics to validate the time/$?
    5. 5. LOT’S OF TOOLS OUT THERE BEWARE OF THE SILOs AVOID USING (ONLY) THE NEW SHINEY FREEBIES @alanchumley / @CARMA_Tweets Would you like your job performance (read: raise/bonus) to be based only on a Klout Score, for example? <ul><li>Isolated / silo’d approach </li></ul><ul><li>No one tool on it’s own is sufficient—not even one paid tool </li></ul><ul><li>Have to connect the dots with a more relational, systems approach </li></ul>Don’t tout or shout out about Klout: A blunt counting instrument Ignores context / topical relevance
    6. 6. <ul><li>It’s not coverage </li></ul><ul><li>It’s not static </li></ul><ul><li>There isn’t an ‘audience’ of consumers </li></ul><ul><li>Prosumers not just consumers: </li></ul><ul><ul><li>simultaneously receiving, interpreting, co-opting, re-appropriating and creating content </li></ul></ul><ul><li>It’s more many-to-many dialogue than one-to-many monologue </li></ul><ul><li>So, traditional off line ways of measuring ‘coverage’ don’t cut it </li></ul><ul><li>They don’t adequately account for the multi-dimensionality and dynamic nature of a conversation and a community of interest in a network </li></ul><ul><li>We need to think bigger </li></ul>REMEMBER: IT’S SOCIAL MEDIA @alanchumley / @CARMA_Tweets
    7. 7. THINK BIGGER & BROADER MOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS @alanchumley / @CARMA_Tweets Systems or Networked Thinking: Interdependent relationships between those in a network Social Capital: Connections within and between those in a social network as a source of civic engagement among a community leading to cohesion; that has ‘value’ Michael Kelly
    8. 8. THINK BIGGER / INTEGRATE / CORRELATE <ul><li>Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all that Way </li></ul><ul><li>Think about all customer-facing touch points </li></ul><ul><li>Understand PR / Marketing / Business Metrics </li></ul><ul><li>Integrate Cross-Discipline </li></ul><ul><li>Report on the Critical Few </li></ul><ul><li>Focus on /correlate with Outcomes </li></ul><ul><li>Close the loop / connect the dots </li></ul>Source: Weber-Shandwick Measurement & Strategy Practice <ul><li>It’s not a ‘keep-your-job’ or ‘buy-a-pool-with-your-bonus’ report card </li></ul><ul><li>A data-driven approach / evidence-based communications </li></ul>Source: Oliver Blanchard, Social Media ROI @alanchumley / @CARMA_Tweets Michael Kelly
    9. 9. The 7Cs of SOCIAL MEDIA MEASUREMENT THINKING MORE LIKE A SOCIOLOGIST THAN A PUBLICIST CONVERSATIONS & COMMUNITIES VS. COVERAGE @alanchumley / @CARMA_Tweets Counting What some call search, syndication and site; stuff we can count like traffic, popularity, fans, followers, retweets, etc. Content Analysis that is. Using content analysis, ideally a hybrid of some automation for volume and speed and human analysts for context, nuance and depth to look for how your brand, messaging, topics, issues, stakeholder voices are portrayed; the inclusiveness, accuracy and tone of the conversations. Conversations And conversationships recognizing that social media is not strictly another communications channel through which to get word out or reach and influence. It is more about genuinely engaged dialogue than a simple corporate monologue; not only a vertical, one-to-many conversation between an organization and stakeholders but a lateral, many-to-many conversation among stakeholders. Cohesion / Capital The extent to which folks are agreeing with you, your position, your brand essences. The extent to which they are agreeing with each other and coalescing around and advocating a core theme, idea, or some call to action either in support or opposition to your organization’s position. Community The extent to which a core group of people with common interests are gathering and growing. Measuring the change in community size, and volume and nature of chatter over time Connectedness A look at how inter-connected the highly engaged/key influencers/advocates are and how centrally located they are in both the network and dialogue. Conversion Stay tuned. More to follow.   
    10. 10. THE 6Ps OF INFLUENCE MORE THAN JUST REACH / FREQUENCY RELEVANCY @alanchumley / @CARMA_Tweets Popular Visible, vocal, has a substantial following, reach.  In-bound links, trackbacks, subscribers, bookmarks, followers, friends, views, listens, saves, downloads, etc. Polarized in tone   Neutrality does little to drive influence way or the other.  A clearly positive or negative view will polarize readers/followers and is more likely to drive cohesion and mobilize advocates and have those advocates coalesce around a core theme, idea, or call to action.     Prolific / Relevant / Frequent   Raw author contribution and # of on-topic, related posts Prominent / Authoritative   Are they an idea starter or spreader; source or spider?  They may be prolific but are they prominent?  Are they highly inter-related, inter-connected, and centrally located in the network?  How engaged is this person’s following in a dialogue?  How much dialogue is there and what is its nature?   Here we need to recognize, though, that authority is contextual and topical.  One might be an authority on PR measurement but not on 18th century Russian literature.  Promoter / Loyal Advocate   How many of the followers/commentators active contributors advocating, endorsing, advancing (or the opposite) your position?  Are they adding links, tags.  Is the nature of the language they are using inter-connective, expanded, clarifying, reinterpreting?  RTs, digs, fans, votes, buzz-ups, up/downloads, shares, likes, invites, favorites, embeds.  (More active than the metrics in popularity) Profitable  (conversion / value) Someone who drives others to some form of value or conversion
    11. 11. ‘ REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE!’ NOT ALL CONVERSATIONS ARE RELVANT @alanchumley / @CARMA_Tweets Everybody Unsegmented fans, followers, etc. The people we really care about Influential Authoritative Advocates Cascaders Wouldn’t you rather ‘reach’ 100% of only those that really matter vs. 10% of everybody? B2B space: your influential community might only be 400 people
    12. 12. <ul><li>Objectives </li></ul><ul><li>Visibility </li></ul><ul><li>Visits </li></ul><ul><li>Volume </li></ul><ul><li>Velocity </li></ul><ul><li>Awareness </li></ul><ul><li>Message delivery </li></ul><ul><li>Link love </li></ul><ul><li>Dialogue vs. Monologue </li></ul><ul><li>Generate dialogue / comments </li></ul><ul><li>Feedback </li></ul><ul><li>Using, sharing, pass-a-long </li></ul><ul><li>Embedding of interactive content </li></ul><ul><li>Diggs, Delicious tags, stumbles, etc. </li></ul><ul><li>Votes </li></ul><ul><li>Etc., etc., etc. </li></ul><ul><li>Campaign Types </li></ul><ul><li>Push (get bloggers to review / write) </li></ul><ul><li>Pull (drive visitors on-line) </li></ul><ul><li>CSR </li></ul><ul><li>Product Launch </li></ul><ul><li>Product Engagement </li></ul><ul><li>Marketing </li></ul><ul><li>Blogger Outreach </li></ul><ul><li>Advocacy </li></ul><ul><li>Issues Management </li></ul><ul><li>Community building </li></ul><ul><li>Customer relations </li></ul><ul><li>Thought leadership </li></ul><ul><li>Executive profiling </li></ul><ul><li>Etc., etc., etc. </li></ul>VARIETY OF OBJECTIVES & CAMPAIGN TYPES (NON-EXHAUSTIVE) @alanchumley / @CARMA_Tweets
    14. 14. COMPONENTS OF AN INDEX ? <ul><li>Based on an assumption that the components above are drivers of on-line influence, potential action </li></ul><ul><li>They can be measured individually, but it’s important to measure them collectively as they are inter-related </li></ul><ul><li>The index is intended as a measure of potential to influence not actual real influence—that is another research project all together. </li></ul><ul><li>All components are important (but not equal) drivers of potential influence on-line </li></ul>Popularity + Relevance + Authority + Presence + Engagement + Advocacy/Loyalty + Cascade x (+/-) Sentiment = Influence Potential @alanchumley / @CARMA_Tweets
    16. 16. <ul><li>Each of the above indicators can be adjusted to account for different types of campaigns & contexts </li></ul><ul><li>The weights would change . The scores don’t. </li></ul><ul><li>For an organization, new to social media, perhaps they might only want to establish an on-line presence </li></ul><ul><li>So we’d dial or slide the weights of all but the popularity indicator down to zero </li></ul>AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES @alanchumley / @CARMA_Tweets Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking.
    17. 17. INDICATORS DEFINED <ul><li>Cascade </li></ul><ul><ul><li>Citation analysis / ripple index </li></ul></ul><ul><ul><li>Network analysis </li></ul></ul><ul><ul><li>Volume & velocity of message spread across the network </li></ul></ul><ul><ul><li>Inter-connectedness / inter-relationships among: </li></ul></ul><ul><ul><ul><li>all </li></ul></ul></ul><ul><ul><ul><li>Only the influentials </li></ul></ul></ul><ul><li>Advocacy (active) </li></ul><ul><ul><li>Are social media users advancing our cause? </li></ul></ul><ul><ul><li>% containing links </li></ul></ul><ul><ul><li>% active contributors </li></ul></ul><ul><ul><li>Tags added </li></ul></ul><ul><li>Engagement (more coming) </li></ul><ul><ul><li>Dialogue? 2-way? How much? How good? </li></ul></ul><ul><ul><li>How many commentators? Repeats? Nature of the language </li></ul></ul><ul><ul><li>Network analysis: inter-connected/related </li></ul></ul><ul><li>Entity Presence </li></ul><ul><ul><li>Present? Prominent? Frequent? </li></ul></ul><ul><ul><li># of + mentions </li></ul></ul><ul><ul><li>Competitive share of original post </li></ul></ul><ul><ul><li>Competitive share of resulting discussion </li></ul></ul><ul><ul><li>Attribution recognition </li></ul></ul><ul><li>Authority </li></ul><ul><ul><li>Of both poster and commentators </li></ul></ul><ul><ul><li>Idea starters or spreaders? </li></ul></ul><ul><ul><li>Degree of inter-connectedness / inter-relatedness </li></ul></ul><ul><ul><li>Connected but respected? </li></ul></ul><ul><ul><li>How central is the influencer to the network? </li></ul></ul><ul><ul><li>Following: follower </li></ul></ul><ul><ul><li>RTs/1000 followers </li></ul></ul><ul><li>Relevance </li></ul><ul><ul><li>On topic? </li></ul></ul><ul><ul><li>Author contribution: how often / prolific? </li></ul></ul><ul><ul><li>Topical / textual / temporary </li></ul></ul><ul><li>Sentiment (more coming) </li></ul><ul><ul><li>Multi-level, multi-author tone </li></ul></ul><ul><ul><li>Post-level, theme level, sentence level </li></ul></ul><ul><ul><li>Gaps in tone in originating content and resulting discussion </li></ul></ul><ul><li>Popularity (passive) </li></ul><ul><ul><li>‘ Reach’ ? </li></ul></ul><ul><ul><li>Voice of the consumer stuff we can count </li></ul></ul><ul><ul><li>Links, votes, shares, recommends, subscribers, registrations, tweets, RTs, followers, fans, friends, diggs, shares, likes, fav’s, upvotes, buzzups, embeds, tags, wish list saves, forwards, invites, etc. </li></ul></ul>@alanchumley / @CARMA_Tweets
    18. 18. <ul><li>It’s a relative measure of the quality of social media coverage : </li></ul><ul><ul><li>One post relative to another and over time </li></ul></ul><ul><ul><li>Many posts relative to a number of other similar clusters of postings </li></ul></ul><ul><li>It is a measure of both output and outtake, but not outcome </li></ul><ul><ul><li>Output: how much and how good? </li></ul></ul><ul><ul><li>Outtake: how engaged, inter-connected, inter-related, volume & velocity of network / community spread </li></ul></ul><ul><ul><li>Outtake: advocacy </li></ul></ul><ul><ul><li>it does not claim to be a measure of outcome or impact </li></ul></ul><ul><li>It’s a proxy for on-line influence ; it’s about potential to influence </li></ul><ul><ul><li>to measure actual influence, we’ve have to survey </li></ul></ul><ul><li>In a diagnostic, benchmarking sense the index can help inform an organization’s social media engagement strategy if tracked over time </li></ul><ul><li>It’s tactical on the fly for mid-campaign course correction and looking back over the long term </li></ul><ul><li>It’s strategic looking forward: helps shape communications recommendations </li></ul><ul><li>This data can be correlated with tangible outcomes </li></ul><ul><li>This data can be plugged into a market mix model to show causality …isolate for PR’s unique contribution to the marketing mix. </li></ul>WHAT DOES THE INDEX INDICATE? @alanchumley / @CARMA_Tweets
    19. 19. (A QUICK ASIDE ON) ENGAGEMENT MONOLOGUE OR DIALOGUE? Source: Olivier Blanchard, Social Media ROI @alanchumley / @CARMA_Tweets Better Not-so-Good Best <ul><li>Consumers vs. Prosumers </li></ul><ul><li>But haven’t ‘consumers’ always been talking to one another? </li></ul><ul><li>Public vs. private conversations </li></ul>Isn’t this a ‘system’ or a network? How could you measure this without a systems / network approach?
    20. 20. (A QUICK ASIDE ON) SENTIMENT AS MARKET RESEARCH PROXY / PREDICTOR <ul><ul><li>Think here about social media monitoring and analysis as: </li></ul></ul><ul><ul><li>Real time situational awareness </li></ul></ul><ul><ul><li>Listening at the point of need </li></ul></ul><ul><ul><li>Listening close to the moment of influence </li></ul></ul><ul><ul><li>A real-time digital form of or proxy for market research and CRM: </li></ul></ul><ul><ul><li>Mining conversations to tease out how people ‘feel’ about products/services/issues/causes and perhaps even: </li></ul></ul><ul><ul><li>their intended (reported) behavior and what motivates them to buy, (as cited in the conversations), and </li></ul></ul><ul><ul><li>their actual behavior tracked via CRM 2.0 and Web 2.0 analytics </li></ul></ul><ul><ul><li>BUT!!: we’re only talking about people online and visible / vocal people online (not the casual lurkers) in public social media spaces </li></ul></ul>@alanchumley / @CARMA_Tweets Source: Crimson Hexagon
    21. 21. SO WHAT? WHAT ABOUT CONVERSION & ROI? 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818
    22. 22. WE’RE BACK TO THE PYRAMID THE TOP THIS TIME @alanchumley / @CARMA_Tweets
    23. 23. OUTCOMES: KINDS OF CONVERSION SETTING THE STAGE FOR CORRELATION <ul><li>Conversion to a tangible like web traffic, sales, donations, awareness, opinion, usage, brand preference, likelihood to try/buy/recommend. </li></ul><ul><li>Could also mean conversion toward any measurable marketing communication or public affairs, issues, advocacy-based objective. </li></ul><ul><li>Here we could / should also be looking at the extent to which the quantity and quality of social media activity correlates with these outputs . </li></ul>@alanchumley / @CARMA_Tweets Micro (steps toward destination) Macro (Destination) Usually an intermediate means to an end The end Non-transactional (Engagement) Transactional Non-financial Financial Get to x clicks in a site, spend x time on site, download, lead generated, watch a video, created account, contacted us etc. Sale, donation, (paid) subscription etc.
    24. 24. 2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITE Where were they? What drove them to your site? What did they do once on your site? What are they ‘worth’ to you? @alanchumley / @CARMA_Tweets
    25. 25. <ul><li>Export Data to Your Web Analytics </li></ul><ul><li>#of posts by date </li></ul><ul><li># posts with x or higher an influence index score </li></ul>SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICS @alanchumley / @CARMA_Tweets Or Integrate web analytics into your social media monitoring platform
    26. 26. LEVELS OF LINKING OUTPUT TO OUTCOMES Source: Olivier Blanchard, Social Media ROI @alanchumley / @CARMA_Tweets Michael Kelly Level of Linkage Notes Just graph ‘er up Plot # of articles vs. web traffic on a graph Don’t do it <ul><li>No indication of relationship other than lines that might trend together </li></ul><ul><li>Could be a fluke </li></ul>Correlation <ul><li>“ R score” </li></ul><ul><li>Some stats </li></ul><ul><li>You can do this in excel </li></ul><ul><li>Pearson’s Product Moment </li></ul><ul><li>Better </li></ul><ul><li>Showing there is some relationship beyond coincidence </li></ul><ul><li>Not ‘proving’ media exclusively responsible for sales </li></ul>Causality <ul><li>Heavy duty stats </li></ul><ul><li>You so can’t do this in excel </li></ul><ul><li>Sophisticated modeling </li></ul><ul><li>Market Mix Modeling </li></ul><ul><li>Multivariate Regression Analysis </li></ul><ul><li>Best </li></ul><ul><li>Isolates (statistically ‘proves’) for PR’s unique contribution to the communications mix </li></ul><ul><li>Ton ‘o data, time, $ </li></ul>
    27. 27. UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs @alanchumley / @CARMA_Tweets Source: Olivier Blanchard, Social Media ROI ROI roi MBA PR Financial Non-financial Macro Micro (not all conversion events are ROI) A mathematical equation that works Usually conceptual only; an equation in words only; can’t actually be calculated; something the industry tosses around too liberally A business metric All-too-often a media or audience metric Direct, clear A loose proxy or an intermediate metric (return on expectation, return on objective, return on engagement)
    28. 28. SOCIAL MEDIA BOOK BAG <ul><li>Social Media Metrics: </li></ul><ul><ul><ul><li>Jim Sterne, (@jimsterne) </li></ul></ul></ul><ul><li>Social Media ROI: </li></ul><ul><ul><ul><li>Olivier Blanchard (@thebrandbuilder) </li></ul></ul></ul><ul><li>Social Media Analytics: Effective Tools for Building, Interpreting & Using Metrics : </li></ul><ul><ul><ul><li>Marshal Sponder (@webmetricsguru) </li></ul></ul></ul><ul><li>Measure What Matters: </li></ul><ul><ul><ul><li>Katie Paine (@kdpaine) </li></ul></ul></ul><ul><li>Using Web Analytics to Measure Impact of Earned Online Media…: </li></ul><ul><ul><ul><li>Seth Duncan ( </li></ul></ul></ul>@alanchumley / @CARMA_Tweets
    29. 29. THANK YOU QUESTIONS? 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 Alan Chumley Senior Vice President @alanchumley / @CARMA_Tweets
    30. 30. APPENDIX 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 Alan Chumley Senior Vice President @alanchumley / @CARMA_Tweets
    31. 31. How are these changing over time? Might be part of many systems / networks? Linking networks to networks @alanchumley / @CARMA_Tweets NETWORK ANALYSIS Source: K. Ognyanova, UCLA, 2010