Your SlideShare is downloading. ×
  • Like
  • Save
Measurement mini workshop
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Measurement mini workshop

  • 3,452 views
Published

 

Published in Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,452
On SlideShare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
3

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Welcome – count down
  • In KD Paine’s decades of experience, the measurement process looks like this – no matter what you’re measuring to get at measuring what matters … The strategy piece includes creating a plan, it might include research (audience research or benchmarking w/peers). For some of you, your pilot might be “prospective” – as part of planning a future campaign or program. Or it might be analyzing the past – something you’ve already implemented but want to go back and measure results/value for planning next reiteration or to do present to senior management to pave the way for more incorporating social media into your program delivery or communications channel. The do part – your tactics, tools – The measure is your data collection – as part of your strategy you define this – it includes analysis – you don’t just collect data, you figure out what it means in terms of improving your results – and then you adapt your plan/strategy and the cycle continues.Make sense? Does this sound familiar? Is this way you measure other programs or communications efforts that do not include social media?
  • These are the basic steps that you will follow for your measurement pilot …. Or any project that involves social media ..The challenge here is that you are not just measuring, but measuring around your practice or strategy.The pilot is making it very simple.
  • Start with your results and incorporate social media into it .. Keep asking to what end? Why is this important? AwarenessReputationRelationshipsActionsDonationsVolunteersBehavior ChangePartner Engagement
  • Theory of Changehttp://www.bethkanter.org/sm-toc/A series of if, then statements that are laid out as a path …This might be a useful approach for those of you working more on policy or program outcomes – and how to integrate social media For example:Kansas Action for Children has helped shape policy that puts children first.
  • What are you going to do to get those results with an integrated social media plan?SMART ObjectivesMeasurement PlanAudience Definition: Target Audience InfluencersEnvironmentMessaging/ContentChannelsBudgetTimeline
  • A strategy is a plan of action designed to achieve a particular objective. Tactics and tools are the means by which a strategy is carried out – these are on the ground methods to executive strategy. Ways to be strategic …Smart Objectiveshttp://www.bethkanter.org/25-smart/How many by when?Results, Capacity, or TacticsGuides, not report cards
  • Key performance indicators are 3-5 metrics that are critical to knowing you have been reach your result or not ….
  • Key Performance IndicatorsMetricshttp://www.mediapost.com/publications/?fa=Articles.showArticle&art_aid=117581
  • Key Performance Indicators: Key performance indicators are the most important metrics to understand how successful your are ..This is the conversation to have with your team …..Metricshttp://www.mediapost.com/publications/?fa=Articles.showArticle&art_aid=117581
  • USA for UNHCR (my client) is a 501(c)(3) headquartered in Washington, D.C. They raise funds and awareness in the United States for the lifesaving work that more than 6,000 staffers of Geneva-based UNHCR undertake for refugees around the world, 24/7.   Given that there are more than 43.3 million refugees worldwide, this is pretty important work… and did I mention lifesaving? (Just look at what’s happening in Somalia right now.) The challenge is that typically refugees are so far removed from our “normal” lives that we don’t automatically think of giving to organizations that work to support them.USA for UNHCR created the Blue Key campaign as a way to drive awareness of this global issue in the US. The $5 blue key pin or pendant symbolizes the one thing most of us have and that refugees don’t: a key to their own home.My role is to work with USA for UNHCR on building out the digital and social media aspects of the campaign, particularly from an online community-building point of view. That is where the Blue Key Champion strategy came in; to bring on board US-based bloggers who are active online, and cause-passionate, to help spread the word and, hopefully, get more keys ordered.
  • The Blue Key site was only launched in December 2010, and its social/digital aspects were relatively new, so there was not a lot of data to base KPIs on. Overall, when we went into the first phase of the campaign, we had two goals: to secure at least three Blue Key Champions, and  to get 6,000 keys ordered between May 9 and June 20 (World Refugee Day).http://www.bethkanter.org/un-blue-key/
  • http://www.flickr.com/photos/tjblackwell/5659432136/sizes/m/in/photostream/Cost to implement your social media pilot – including measurement time! We know that most of the software/platforms are free, so the cost is going to be an investment in staff time. What is the level of time?What is the opportunity cost?Are you adding resources or shifting them?What are the consequences?How can you be as efficient as possible in your execution time? Data collection time?You’ll need this for reporting … and also helps you get better and more efficient
  • http://www.flickr.com/photos/kaptainkobold/80911944/sizes/o/in/photostream/Your measurement tools are the techniques you will use to collect data – the key performance indicators – These are covered in a later session. Remember – any tool is useless unless it covers the sort of data that help you evaluate progress towards to your goalsContent analysis of social or traditional media- Primary audience via online, mail, or phone surveys- AnalyticsAffordable ….
  • http://www.flickr.com/photos/theimagegroup/369893824/Measurement is a continuous improvement process – feedback loopsAny measurement to work, you need to assess results, make changes, and see if those changes had impact, make more changes, and so on. Valid, actionable conclusionsAvoid focusing on only the most exciting, look for failures …You need a regular reporting schedule – not just do a data dump at the end – important for buy-in
  • These are the basic steps that you will follow for your measurement pilot ….
  • These are the basic steps that you will follow for your measurement pilot ….
  • http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/
  • http://www.flickr.com/photos/casch/375275440/sizes/o/in/photostream/Fear that it is “not fun” Creating is more appealing then analysis to most people who work in nonprofits. Sadly, many people are more likely to build instead of measure even if they are building on a fault line.  That comes as no surprise because most people don’t decide to go into non-profit work because they want to sit and look at spreadsheets all day. Even if there is a staff position for analytics, it tends to be in finance not communications. And not all communications departments invest in professional development in measurement for staff members responsible.  1/in/photostream/
  • http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/Fear of consequences The nonprofit world can be a scary place in a fragile economy.No one wants to take risks if the results could lose you a promotion or worse, lose your job. So inherently measuring results is a scary prospect.  What if we discover that our campaign didn’t get the results we thought it would or even worse, that our precious time was wasted.   
  • http://www.flickr.com/photToo often, there is a fear that by setting objectives, you will be held accountable for some else’s unreasonable expectations. This is the result of “SMART” objectives – which we will discuss in depth in the next chapter. Not setting measurable objectives is akin to running a marathon in an earthquake blindfolded. http://www.bethkanter.org/25-smart/os/alistairhall/5223945711/in/photostream/
  • http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/There is a common misperception that valid measurement requires highly trained specialists. Or if it is done in house, it is considered time-consuming and just one more chore on a seemingly endless list. However, if nonprofits match their data collection approaches to their capacity, chances are measurement will save time and resources. It is better to start small with a few data points and strategically add metrics to the spreadsheet dashboard. Nonprofits need to think of the measurement task as training for marathon.   If you are just starting to get into shape, you need to build up your strength before you run a 35-mile marathon.  http://www.flickr.com/photos/josephrobertson/127758523/sizes/l/in/photostream/
  • http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/Many nonprofits default to the “measure everything” approach on the mistaken belief that more data (or even bogus data (is better than nothing. Worse they collect data “just in case” Either way, these results in bloated spreadsheets to which no one pays attention.  so a data-driven organization needs to be comfortable in just saying “No!” to too much data. In a recent interview, Jodi Nelson, who heads up the Bill & Melinda Gates Foundation’s Impact Planning and Improvement Unit, said, “Even when you’ve figured out the metrics, it doesn’t necessarily mean that the data you collect end up getting used. It may sound intuitive, but the key is figuring out how to guarantee the data are both relevant and used. You’d be amazed at how much data collection goes on in the world that produces reports that are never read or used to do anything differently. “http://www.alliancemagazine.org/node/3699
  • 1. Visualize success and failure2. Spend more time identifying what you want to measure, not how to measure it 3. Measure in context – don’t ever collect data unless you have SMART objectives and a benchmark to measure it against.4. Don’t wait until the end of the pilot to gather and analyze your data. Build measurement in from the start and then collect in regular weekly or monthly increments.5. Don’t ever just shovel data over the fence and onto your executive director’s desk. Share high level insights, make recommendations that can spark ideas on improvement6. Less is more. Discipline yourself to measure only one objective or one channel and don’t collect more than data points.7 . Start with a small measurement pilot that helps your organization create a habit of collecting, analyzing, and applying data. We describe in this great detail in the next sections.

Transcript

  • 1. Measurement Trick or Treat
    Beth Kanter, Beth’s Blog -- October, 19, 2011
  • 2.
  • 3. KD Paine
    Measurement
    Basics
  • 4. KD Paine’s Basic Steps
    Define ResultsStrategy BenchmarksMetrics – KPICosts
    Select Right Tool To Collect DataTurn Data in Action
  • 5. Define Results
    Keep asking to what end?
    AwarenessReputationRelationshipsActionDonationsVolunteersBehavior Change
  • 6. Theory of Change Lite: To What End?
    Kidds
  • 7. Key Result: Kids get health care
    Gets better at communication, proactive planning, partner engagement, message discipline and social media integration
    More and better partners, more and better relationships with reporters and more or better policy maker relationships
    Communications strategy has more impact
    More attention from policymakers
    Policies are adopted
    Kids get health care coverage
  • 8. StrategySMART ObjectivesMeasurement PlanAudience Definition:
    Target Audience InfluencersEnvironmentMessaging/ContentChannelsBudgetTimeline
  • 9. SMART objectives
    1.How many?
    2. By when?
  • 10. Integrated Social Media StrategyFacebook Measurement Pilot
  • 11. Target Audience: AdvocatesIntermediary Audience: Policy Makers
    Advocates send 200 emails, make 300 phone calls, and post on FB page to endorse an issue related to ACAIdentify and establish relationships with “influencers” (lawmakers, parents, etc) through Facebook so they spread stories of Children’s Health Care by 1/2012
    Increase number of fans for Facebook page by 10% by 2012
    Increase engagement with Facebook fans to 3 comments per post by 2012
    Repurpose policy paper content into series of regular Facebook posts by 2012
    Two aligned partners collaborate on an online event on Facebook to draw attention to issue by 2012
  • 12. Benchmarks
    "The greatest danger for most of us is not that our aim is too high and we miss it but that it is too low and we reach it." Michelangelo
  • 13.
  • 14.
  • 15. DYI Benchmark Study of Peers
  • 16. Use industry studies
  • 17. Define Your Key Performance Indicators
    What are the metrics you will report with?
  • 18. There is no shortage of metrics you could use …….
    27.  Social bookmarks
    28.  Subscriptions (RSS, podcasts, video series)
    29.  Pageviews (for blogs, microsites, etc)
    30.  Effective CPM based on spend per impressions received
    31.  Change in search engine rankings for the site linked to through social media
    32.  Change in search engine share of voice for all social sites promoting the brand
    33.  Increase in searches due to social activity
    34.  Percentage of buzz containing links
    35.  Links ranked by influence of publishers
    36.  Percentage of buzz containing multimedia (images, video, audio)
    37.  Share of voice on social sites when running earned and paid media in same environment
    38.  Influence of consumers reached
    39.  Influence of publishers reached (e.g., blogs)
    40.  Influence of brands participating in social channels
    41.  Demographics of target audience engaged with social channels
    42.  Demographics of audience reached through social media
    43.  Social media habits/interests of target audience
    44.  Geography of participating consumers
    45.  Sentiment by volume of posts
    46.  Sentiment by volume of impressions
    47.  Shift in sentiment before, during, and after social marketing programs
    48.  Languages spoken by participating consumers
    49.  Time spent with distributed content
    50.  Time spent on site through social media referrals
    1.     Volume of consumer-created buzz
    2.     Amount of buzz 3.     Shift in buzz over time
    4.     Buzz by time of day
    5.     Seasonality of buzz
    6.     Competitive buzz
    7.     Buzz by category
    8.     Buzz by social channel 9.     Buzz by stage in purchase funnel
    10.  Asset popularity
    11.  Mainstream media mentions
    12.  Fans
    13.  Followers
    14.  Friends
    15.  Growth rate of followers friends
    16.  Rate of pass-along
    17.  Change in rates
    18.  Second-degree reach
    19.  Embeds / Installs
    20.  Downloads
    21.  Uploads
    22.  User-initiated views
    23.  Ratio of embeds to views
    24.  Likes / favorites
    25.  Comments
    26.  Ratings
    Source: 100 Ways to Measure Social Media by David Berkowitz
  • 19. There is no shortest of metrics you could use …….
    75.  Customers assisted
    76.  Savings per customer assisted through direct social media interactions compared to other channels (e.g., call centers, in-store)
    77.  Savings generated by enabling customers to connect with each other
    78.  Impact on first contact resolution (FCR) (hat tip to Forrester Research for that one)
    79.  Customer satisfaction
    80.  Volume of customer feedback generated
    81.  Research & development time saved based on feedback from social media
    82.  Suggestions implemented from social feedback
    83.  Costs saved from not spending on traditional research
    84.  Impact on online sales
    85.  Impact on offline sales
    86.  Discount redemption rate
    87.  Impact on other offline behavior (e.g., TV tune-in)
    88.  Leads generated
    89.  Products sampled
    90.  Visits to store locator pages
    91.  Conversion change due to user ratings, reviews
    92.  Rate of customer/visitor retention
    93.  Impact on customer lifetime value
    94.  Customer acquisition / retention costs through social media
    95.  Change in market share
    96.  Earned media's impact on results from paid media
    97.  Responses to socially posted events
    98.  Attendance generated at in-person events
    99.  Employees reached (for internal programs)
    100.  Job applications received
    51.  Method of content discovery
    52.  Clicks
    53.  Percentage of traffic generated from earned media
    54.  View-throughs
    55.  Number of interactions
    56.  Interaction/engagement rate
    57.  Frequency of social interactions per consumer
    58.  Percentage of videos viewed
    59.  Polls taken / votes received
    60.  Brand association
    61.  Purchase consideration
    62.  Number of user-generated submissions received
    63.  Exposures of virtual gifts
    64.  Number of virtual gifts given
    65.  Relative popularity of content
    66.  Tags added
    67.  Attributes of tags 68.  Registrations from third-party social logins
    69.  Registrations by channel (
    70.  Contest entries
    71.  Number of chat room participants
    72.  Wiki contributors
    73.  Impact of offline marketing/events
    74.  User-generated content created that
    75.  Customers assisted
    What matters most?
    Source: 100 Ways to Measure Social Media by David Berkowitz
  • 20.
  • 21.
  • 22.
  • 23. Cost
  • 24. The right tool for the job
    Content analysis of social or traditional media- Primary audience via online, mail, or phone surveys- Analytics
  • 25. GRIST.ORG
    KPI: Footprint: The reach of their activities, both online and offline
    ViewsGoogle Analytics
    Engagement: Readers engage with their content
    Comments, Virility, RetweetsChart BeatFacebook InsightsTwitter Crowd
    Individual Behavior Change: Impact on users behaviors, purchase decisions, and daily lives that are in line with sustainability
    Questions about habits
    Survey Monkey
    Societal Change: Impact on society, policy discussions, and conversations that advance sustainable practices.
    Anecdotal stories
  • 26. Turn Data in Action
  • 27.
  • 28. KD Paine’s Basic Steps
    Define ResultsStrategy BenchmarksMetrics – KPICosts
    Select Right Tool To Collect DataTurn Data in Action
  • 29. Measure Value
    Direct RevenueCost Savings
    Traffic/Search Engine Rankings
    Social Capital
  • 30. Why does calculating give us the collywobbles?
    The 5 Deadly Fears That Get in the Way of Good Measurement
  • 31. Fear that isn’t fun
  • 32. Fear of Consequences
  • 33. Fear of objectives
  • 34. Fear of being overwhelmed
  • 35. Fear of the data swamp
  • 36. The 7 Habits of Highly Effective Nonprofits
    1. Visualize success and failure
    2. Spend more time identifying what you want to measure, not how to measure it
    3. Measure in context – don’t ever collect data unless you have SMART objectives
    4. Don’t wait until the end of project to collect data
    5. Don’t ever just shovel data over the fence and onto your executive director’s desk
    6. Less is more
  • 37. Thank you
    http://www.bethkanter.org
    http://socialmedia-strategy.wikispaces.com/
    http://www.facebook.com/beth.kanter.blog