Tips and Techniques to Measure Social Media Measurement 2009


Published on

This is my social media measurement deck from the IPR Measurement Summit 09

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Tips and Techniques to Measure Social Media Measurement 2009

  1. 1. Social Media Measurement Techniques and Tips  October 14 , 2009Katie Delahaye PaineCEOkdpaine@kdpaine.comwww.kdpaine.comhttp:/kdpaine.blogs.comMember, IPR Measurement<br />
  2. 2. What Matters? <br />To P&G: Engagement<br />To the Humane Society: Donations<br />To ComCast: Happier customers <br />To Best Buy: Better informed employees<br />To WMUR: Faster, more complete, more relevant stories <br />To Dell: Sales<br />To Molson: Better messaging <br />
  3. 3. What Doesn’t Matter?<br />AVEs<br />Eyeballs<br />HITS (How Idiots Track Success)<br />Couch Potatoes<br /># of Twitter Followers (unless you’re a celebrity)<br /># of Facebook Friends/Fans (unless they donate money) <br />Page 3<br />
  4. 4. Page 4<br />You are a party planner, not a communicator <br />21st Century <br />Old School <br />
  5. 5. Page 5<br />Social Media renders everything you know about measurement obsolete<br />Old School PR<br />21st Century Role of PR <br />The definition of timely has changed<br />The definition of reach has changed<br />GRPs & Impressions are impossible to count (an irrevelvant) in social media <br />The definition of success has changed<br />The answer isn’t how many you’ve reached, but how those you’ve reached have responded <br />
  6. 6. Some really scary numbers* if you’re an ad agency <br />More respondents spend more time on daily personal Intent usage than watching TV <br />53% of DVR owners watch at least 50% of content on replay – skipping ads all together <br />Consumer Internet ad spend outpaces TV spend by 3X<br />26% of US respondents have already contributed content to social networking sites. <br />32% said they follow recommendations from friends<br />2 out of 3 ad execs expect ad revenue to shift from impression-based to impact based metrics within three years <br />*The End of Advertising as we Know it , IBM 2009<br />
  7. 7. Signs that it’s the end of measurement as we know it<br />11 Moms make a bigger difference than 11 million <br />1 person on Twitter changed the reputation of Comcast <br />1 CEO’s blog is changing the face of healthcare in Boston<br />Facebook USERS translated the site from English to Spanish via a Wiki in less than 4 weeks and cost Facebook $0 <br />Dell has made more money on Twitter than Twitter has <br />Ex-employee networks are helping companies lower costs and speed response times <br />The Epping, NH police department is on Twitter <br />
  8. 8. The New Rules of Communications <br />You aren’t in control and never have been<br />There is no market for your message <br />You become what you measure<br />She/he with the most data wins<br />Behind every Tweet or Post is a person <br />Empower employees, rely on customers<br />Enable the conversations—it’s going on, with or without you<br />Spin is dead, long live transparency – you can’t fake it so be who you are and see who is pleased <br />Crowdsourcing will beat outsourcing every time <br />
  9. 9. The Engagement Decision Tree<br />
  10. 10. Goals for Social Media <br />Marketing/leads/sales/<br />Mission/safety/civic engagement<br />Relationship/reputation/positioning <br />To fix this<br />Or get to this <br />
  11. 11. Goals drive metrics, metrics drive results <br />11<br />Goal<br />Metrics<br />
  12. 12. Change the conversation, improve your reputation <br />Improve your reputation<br />Listen first, then respond<br />Stop doing stupid things <br />
  13. 13. Negative coverage over time <br />
  14. 14. Goals, Actions and Metrics <br />
  15. 15. The 7 steps to Social Media ROI <br />Define the “R” – Define the expected results?<br />Define the “I” -- What’s the investment?<br />Understand your audiences and what motivates them <br />Define the metrics (what you want to become) <br />Determine what you are benchmarking against<br />Pick a tool and undertake research<br />Analyze results and glean insight, take action, measure again<br />
  16. 16. Step 1: Define the “R” <br />What return is expected? – Define in terms of the business or mission.<br />What were you hired to do? What difference are you expected to make? <br />If you are celebrating complete 100% success a year from now, what is different about the organization?<br />If your department was eliminated, what would be different? <br />16<br />
  17. 17. Step 2: Define the “I” <br />What is the investment? <br />Personnel<br />Agency compensation<br />Senior Staff time<br /> Opportunity cost<br />Raw costs/hr costs vs material costs. <br />17<br />
  18. 18. Step 3: Define your audiences and how you impact them<br />There is no “audience.” There are multiple constituencies <br />Should you blog or Twitter? Don’t ask me, ask your customers <br />List every stakeholder<br />Where do they go for information?<br />What’s important to them?<br />What is the benefit of having a good relationship with that stakeholder group?<br />What’s important to them?<br />Where do they go for information?<br />What do you want them to know? <br />Understand your role in getting the audience to do what you want it to do<br />Raise awareness<br />Increase preference<br />Increase engagement<br />18<br />
  19. 19. Step 4: Define your Key Performance Indicators (KPIs)<br />19<br />The Perfect KPI<br />Gets you where you want to go (achieves corporate goals)<br />Is actionable<br />Continuously improves your processes<br />Is there when you need it<br />KPIs should be developed for: <br />Your own properties<br />Different tactics<br />Other influential sites <br />
  20. 20. Step 4: Define your Key Performance Indicators (KPIs) carefully because you become what you measure<br />Cost savings<br />Efficiency<br />Cost per message communicated<br />Cost per new lead/customer acquired<br />Productivity: <br />Increase in employee engagement/morale<br />Lower turnover/recruitment costs<br />Engagement: <br />Ratio of posts to comments<br />% of repeat visitors<br />% of 5+min visitors<br />% of registrations<br />Trust:<br />Improvement in relationship /reputation scores with customers and communities (Loyalty/Retention)<br />Thought leadership: <br />Share of quotes<br />Share of opportunities <br />Message penetration<br />Positioning on key issues<br />Improvement in favorable/unfavorable ratio<br />Improvement in Optimal Content Score (OCS)<br />20<br />
  21. 21. KPIs for External blogs and other Consumer Generated Media <br />Share of positioning<br />Share of rants vs. raves<br />Share of positives/negatives<br />Share of visibility<br />Share of quotes<br />Share of brand benefits mentioned<br />Types of conversations<br />Engagement – ratio of posts to comments <br />Optimal content score <br />
  22. 22. Revenue KPIs <br />Cost savings<br />Cost per click thru, downloads, engagement vs other marketing channels<br />Cost per message communicated vs other channels<br />Lifetime value of engagement<br />Cost per customer acquisition <br />
  23. 23. Engagement metrics <br />% increase or decrease in unique visits <br />In the past  month,  what % of all sessions represent more than 5 page views <br />% of sessions that are greater than 5 minutes in duration <br />% of visitors that come back for more than 5 sessions <br />% of sessions that arrive at your site from a Google search, or a direct link from your web site or other site that is related to your brand <br />% of visitors that become a subscriber <br />% of visitors that download something from the site <br />% of visitors that provide an email address<br />Ratio of posts to comments <br />Courtesy of Eric Peterson<br />
  24. 24. For all institutions, most postings were simply making an observation or distributing media. <br />Page 24<br />cx<br />
  25. 25. Share of conversation vs share of engagement <br />Page 25<br />Share of Engagement by Subject <br />-<br />,External Blogs<br />Share of Subject<br />Students<br />23.6%<br />33.2%<br />22.1%<br />21.1%<br />Staff<br />100.0%<br />Research, Social Sciences<br />1<br />4<br />1<br />Research, Social Sciences<br />4.4%<br />95.6%<br />Campus Life<br />Research, Physical Sciences<br />1<br />38.3%<br />2.3%<br />31.0%<br />28.4%<br />Research, Other<br />Institution, Overall<br />2<br />1<br />3<br />Research, Life Sciences<br />13.0%<br />20.8%<br />13.0%<br />53.2%<br />Policies<br />2<br />Research, Earth Sciences<br />86.8%<br />13.2%<br />Research, Agriculture<br />4<br />Research, Agriculture<br />100.0%<br />Projects, Non<br />-<br />Research<br />Other<br />28.6%<br />28.6%<br />28.6%<br />14.2%<br />1<br />Policies<br />100.0%<br />Legal News<br />Peer 1<br />1<br />2<br />Partnerships<br />Michigan State<br />Admissions<br />1<br />1<br />Peer 1<br />Other<br />Peer 2<br />Staff<br />Michigan State<br />1<br />Legal News<br />43.3%<br />56.7%<br />Peer 3<br />Inventions<br />Peer 2<br />Research, Life Sciences<br />1<br />1<br />2<br />1<br />3<br />Peer 4<br />Institution, Overall<br />5.8%<br />94.2%<br />Peer 3<br />Alumni Topics<br />1<br />1<br />Financials<br />68.7%<br />12.5%<br />18.8%<br />Peer 4<br />Financials<br />2<br />1<br />2<br />Faculty<br />15.3%<br />34.9%<br />6.3%<br />43.5%<br />Projects, Non<br />-<br />Research<br />Events<br />1<br />1<br />1<br />2<br />Courses<br />28.6%<br />71.4%<br />Research, Earth Sciences<br />1<br />2<br />2<br />Community Relations<br />Courses<br />1<br />2<br />Campus Life<br />Research, Physical Sciences<br />3<br />2<br />4<br />6<br />Alumni Topics<br />96.8%<br />3.2%<br />Admissions<br />Students<br />33.3%<br />66.7%<br />5<br />2<br />1<br />7<br />Faculty<br />2<br />6<br />2<br />2<br />6<br />0%<br />10%<br />20%<br />30%<br />40%<br />50%<br />60%<br />70%<br />80%<br />90%<br />100%<br />0<br />2<br />4<br />6<br />8<br />10<br />12<br />14<br />16<br />18<br />20<br />
  26. 26. The vast majority of discussion in external blogs is neutral.<br />Page 26<br />
  27. 27. Emerging benchmarks <br />Past Performance<br />Think 3<br />Peer<br />Underdog nipping at your heels<br />Stretch goal <br />Whatever keeps the C-suite up at night<br />Step 5: Define your benchmarks<br />27<br />
  28. 28. Non-Profit industry benchmarks in social media <br />28<br />
  29. 29. Overview of Key Metrics<br />Peer 1 was the competitive leader in all but YouTube, where Peer 4 and Peer 3 led.<br />Actions attributed to individuals were responsible for most content, except on YouTube.<br />
  30. 30. Few subjects appear across all forms of social media, so tailor outreach accordingly <br />
  31. 31. Benchmarks put numbers in perspective<br />Page 31<br />
  32. 32. Step 6: Pick a tool <br />Content Analysis<br />Survey<br />Web Analytics<br />
  33. 33. Step 6: Selecting a measurement tool<br />33<br />
  34. 34. Content Analysis requires: <br />A content source: <br />Google News/Google Blogs, RSS feeds<br />Technorati, Social Mention, Twazzup, <br />Cyberalert, CustomScoop, e-Watch<br />Radian 6, Techrigy, Sysymos, Visible Technologies, Scout Labs<br />34<br />
  35. 35. A way to analyze content <br />Automated vs. Manual <br />Census vs random sample<br />The 80/20 rule – Measure what matters because 20% of the content influences 80% of the decisions<br />Dashboards to aggregate data <br />35<br />
  36. 36. A coding methodology<br />Tonality<br />What messages were communicated<br />How you’re positioned on key issues<br />Dominance/Prominence/Visibility<br />Subject of the article/posting<br />Who was quoted?<br />Products, events, initiatives, battles mentioned<br />
  37. 37. Standard classifications of discussion<br /><ul><li>Responding to criticism
  38. 38. Giving a shout-out
  39. 39. Making a joke
  40. 40. Making a suggestion
  41. 41. Making an observation
  42. 42. Offering a greeting
  43. 43. Offering an opinion
  44. 44. Putting out a wanted ad
  45. 45. Rallying support
  46. 46. Recruiting people
  47. 47. Showing dismay
  48. 48. Soliciting comments
  49. 49. Soliciting help
  50. 50. Starting a poll
  51. 51. Validating a position
  52. 52. Acknowledging receipt of information
  53. 53. Advertising something
  54. 54. Answering a question
  55. 55. Asking a question
  56. 56. Augmenting a previous post
  57. 57. Calling for action
  58. 58. Disclosing personal information
  59. 59. Distributing media
  60. 60. Expressing agreement
  61. 61. Expressing criticism
  62. 62. Expressing support
  63. 63. Expressing surprise
  64. 64. Giving a heads up</li></li></ul><li>Standard classifications of videos<br />Advertisement<br />Animation<br />Demonstration<br />Event/Performance<br />Fiction<br />Film<br />Home Video<br />Instructional Video<br />Interview<br />Lecture<br />Montage<br />Music Video<br />News Broadcast<br />Promotional Video<br />Sightseeing/Tour<br />Slideshow<br />Speech<br />Television Show<br />Video Log<br />
  65. 65. Why an Optimal Content Score? <br />You decide what’s important:<br />Benchmark against peers and/or competitors<br />Track activities against OCS over time <br />Positive: <br />Mentions of the brand<br />Key messages<br />Positioning<br />Visibility<br />Negative <br />Omitted<br />Negative tone<br />No key message<br />39<br />
  66. 66. How to calculate Optimal Content<br />
  67. 67. Building Measures<br />Units of content<br />Overall theme (when many messages are combined)<br />Entire message (e.g. article, blog post, etc.)<br />Message parts (brand mentions, paragraphs, sentences)<br />Types of content<br />Manifest: on the surface<br />Latent: the meaning or interpretation of the content<br />Latent Pattern: meaning determined by surface observations<br />Latent Projective: meaning determined by coder interpretation<br />
  68. 68. Building Measures (cont.)<br />Variables are exhaustive/ mutually exclusive<br />One can always be selected, and only one<br />Types of variables<br />Nominal: categories, “buckets”<br />Brands mentioned, organizations mentioned, messages communicated<br />Ordinal: categories with an order or scale<br />Tonality, prominence, dominance<br />Numbers: number of words (zero means no words), number of brand mentions<br />
  69. 69. Surveys require: <br />A defined sample<br />A list – a way to get to that sample<br />Agreement on what questions you need to answer<br />A survey instrument/questionnaire <br /> A test<br />A way to analyze data<br />SPSS<br />SAS<br />43<br />
  70. 70. Aspects of relationships <br />Control mutuality<br />Trust<br />Satisfaction<br />Commitment<br />Exchange relationship<br />Communal relationship<br />44<br />
  71. 71. Control Mutuality<br />The degree to which parties agree on who has the rightful power to influence one another. Although some imbalance is natural, stable relationships require that organizations and publics each have some control over the other.<br />45<br />
  72. 72. Questions that test Control Mutuality<br />This organization and people like me are attentive to what each other says.<br />This organization believes the opinions of people like me are legitimate.<br />In dealing with people like me, this organization has a tendency to throw its weight around. (Reversed)<br />This organization really listens to what people like me have to say.<br />The management of this organization gives people like me enough say in the decision-making process.<br />46<br />
  73. 73. Measuring Trust<br /> One party’s level of confidence in and willingness to open oneself to the other party. Includes: <br />Integrity: the belief that an organization is fair and just<br />Dependability: the belief that an organization will do what it says it will do<br />Competence: the belief that an organization has the ability to do what it says it will do.<br />47<br />
  74. 74. Questions to measure trust<br />This organization treats people like me fairly and justly.<br />Whenever this organization makes an important decision, I know it will be concerned about people like me.<br />This organization can be relied upon to keep its promises.<br />I believe that this organization takes the opinions of people like me into account when making decisions.<br />I feel very confident about this organization’s skills.<br />This organization has the ability to accomplish what it says it will do.<br />48<br />
  75. 75. Measuring satisfaction<br />The extent to which each party feels favorably toward the other because positive expectations about the relationship are reinforced. A satisfying relationship is one in which the benefits outweigh the costs.<br />49<br />
  76. 76. Questions to Measure Satisfaction<br />I am happy with this organization.<br />Both the organization and people like me benefit from the relationship.<br />Most people like me are happy in their interactions with this organization.<br />Generally speaking, I am pleased with the relationship this organization has established with people like me.<br />Most people enjoy dealing with this organization.<br />50<br />
  77. 77. Measuring commitment<br /> The extent to which each party believes and feels that the relationship is worth spending energy to maintain and promote. <br />51<br />
  78. 78. Commitment<br />I feel that this organization is trying to maintain a long-term commitment to people like me.<br />I can see that this organization wants to maintain a relationship with people like me.<br />There is a long-lasting bond between this organization and people like me.<br />Compared to other organizations, I value my relationship with this organization more.<br />I would rather work together with this organization than not.<br />52<br />
  79. 79. Measuring relationships<br />Exchange Relationship<br /> In an exchange relationship, one party gives benefits to the other only because the other has provided benefits in the past or is expected to do so in the future.<br />Communal Relationship<br /> In a communal relationship, both parties provide benefits to the other because they are concerned for the welfare of the other -- even when they get nothing in return.<br />53<br />
  80. 80. Exchange Relationships<br />Whenever this organization gives or offers something to people like me, it generally expects something in return.<br />Even though people like me have had a relationship with this organization for a long time; it still expects something in return whenever it offers us a favor.<br />This organization will compromise with people like me when it knows that it will gain something.<br />This organization takes care of people who are likely to reward the organization.<br />54<br />
  81. 81. Communal Relationships<br />This organization does not especially enjoy giving others aid. (Reversed)<br />This organization is very concerned about the welfare of people like me.<br />I feel that this organization takes advantage of people who are vulnerable. (Reversed)<br />I think that this organization succeeds by stepping on other people. (Reversed)<br />This organization helps people like me without expecting anything in return.<br />55<br />
  82. 82. How to implement relationship metrics <br />Step 1: Conduct a benchmark relationship study<br />Step 2: Implement PR program<br />Step 3: Conduct a follow up relationship study<br />Step 4: Look at what’s changed <br />
  83. 83. Web Analytics Require: <br />Google Analytics/Web Trends/Omniture<br />Unique URLs<br />Data delivered in parallel with content analysis<br />Ability to correlate and integrate data <br />SPSS/SAS<br />57<br />
  84. 84. Step 7: Analysis - -Research without insight is just trivia<br />Look for failures first<br />Check to see what the competition is doing <br />Then look for exceptional success<br />Compare to last month, last quarter, 13-month average<br />Figure out what worked and what didn’t work<br />Move resources from what isn’t working to what is <br />58<br />
  85. 85. Ask for money<br /> Get Commitment<br /> Manage Timing<br />Influence decisions<br /> Get Outside help<br /> Just Say No<br />Actionable Conclusions<br />59<br />
  86. 86. Overall Comparison of Georgia Tech Social Media Outlets<br />60/17<br /><ul><li>Based on 2007 data, Georgia Tech outperformed its peers in Facebook presence, but significantly lagged peers on other social media.
  87. 87. Post-2007 media monitoring has not included a social media dimension due to funding constraints, but this will be important to trend as feasible in the future.</li></ul>Share of All Coverage<br />Definitions: YouTube: a video sharing site. Social Bookmarking: a site where members can display media they have found on the web. Facebook: a social networking site. Institutional Blogs: blogs hosted and owned by schools studied. External Blog: any blog post that is not hosted by an institution.<br />
  88. 88. Best Practices:<br />Correlations to bottom-line impact<br />Donations<br />Memberships<br />Sign-ups<br />Leads<br />Using SMM for planning<br />Define the time frame, market/topic you want to study<br />Use Google News, Technorati or Radian6 to identify the conversations around the topic <br />Analyze the conversations for type, tone and positioning<br />Look at share of positioning, tone or conversation<br />Benchmarking against your peers<br />Looking at what the best do<br />Setting goals accordingly<br />Use data to persuade recalcitrant spokespeople<br />Social Media in Crisis<br />Listen instantly to a wide range of influencers<br />Identify weaknesses in communications, customer service, or in the product <br />Improve your reputation<br />Listen first, then respond<br />Stop doing stupid things <br />
  89. 89. Using SMM for planning<br />The environmental scan<br />Defining issues in a market<br />Selecting a positioning that works <br />
  90. 90. Benchmarks put numbers in perspective<br />Page 63<br />
  91. 91. Diversity dominates C-M discussions in Social Media <br />Page 64<br />
  92. 92. Where people get the content they share on Facebook<br />Sources of content <br />Genre of content <br />
  93. 93. Understanding brand ownership of online video content<br />Use ownership to signal brand participation<br />Provide alerts for possible brand management issues<br />
  94. 94. 8 ways to do research without a budget<br />Become someone’s research project<br />Involve your board of directors and volunteers<br />Research something that HAS a budget<br />Take advantage of free offers<br />Become a case study<br />Team up with peer organizations<br />Analyze data that already exists<br />Use blogs and social networks to listen to conversations <br />67<br />
  95. 95. Thank You!<br />For more information on measurement, read my blog: or subscribe to The Measurement Standard: <br /><br />For a copy of this presentation go to:<br />Follow me on Twitter: KDPaine<br />Friend me on Facebook: Katie Paine <br />Or call me at 1-603-868-1550<br />