Boston | Geneva | Mumbai | San Francisco | Seattle | Washington FSG.ORGPresentation for:Measuring Social Media Impact:A cl...
2FSG.ORG© 2013 FSGAre communities becoming…More informed?More engaged?Two Questions Guided the Evaluation of theCommunity ...
3FSG.ORG© 2013 FSGWe Utilized the Following Practices inMeasuring Grantees’ Social Media Activities• Understand the goals ...
4FSG.ORG© 2013 FSGUnderstand the Goals of Social Media Activity
5FSG.ORG© 2013 FSGWe Needed a Framework to Understand the Impactof a Diverse Set of Information Projects
6FSG.ORG© 2013 FSGOur Social Media Measurement Focused onProject-Level OutcomesReaching target audienceEngaging target aud...
7FSG.ORG© 2013 FSGWe Began Identifying the Metrics of SuccessIncreasing Reach Increasing Engagement• More (new) people vis...
8FSG.ORG© 2013 FSGThen We Mapped Outcomes to Key PerformanceIndicators (KPIs)Increasing Reach• More (new) people visiting ...
9FSG.ORG© 2013 FSGEntering Data in a Spreadsheet Provided Us withInformation That We Could Use for BenchmarkingTimetrendsG...
10FSG.ORG© 2013 FSGWe Plotted Grantee Data to Better UnderstandProject ReachSource: Anonym zed data from KCIC Year-End Eva...
11FSG.ORG© 2013 FSGSo What about Impact?
12FSG.ORG© 2013 FSGTrack Changes with “Offline” DataDoes filling out an online petition or sharing photos through a nonpro...
13FSG.ORG© 2013 FSGSome Evidence of Real-World EngagementCould Be Found OnlineGrowWNY, Buffalo NYSource photo by Kate Mini...
14FSG.ORG© 2013 FSGWe Collected Data Using More “Traditional”Evaluation Methods to Fully Understand ImpactAdditional Data ...
15FSG.ORG© 2013 FSGIf online “action” is leading to end-game impact?If we are reaching the already converted?What “good” l...
Upcoming SlideShare
Loading in …5
×

Measuring Social Media Impact Lessons from Knight Community Information Challenge

5,234 views

Published on

By Katelyn Mack for Robert Wood Johnson Foundation Social Media Measurement Meeting

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,234
On SlideShare
0
From Embeds
0
Number of Embeds
2,664
Actions
Shares
0
Downloads
21
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Our evaluation examined whether Knight funded grantees were contributing to more INFORMED and ENAGAGED communities. Our evaluation went far beyond SMM We conducted an annual grantee surveyInterviewed project and foundation staffConducted a few case studiesSocial media was one tool that we being used by projects to achieve their goals
  • Some of our key learnings around evaluating information and media projects is found in a publication we entitled IMPACT – which can be found on FSG or Knight’s website.Our SMM activities were used for two primary purposes:To examine the effectiveness of SM strategies across granteesTo look at the impact of SM strategies within particular projects, through more in-depth case studiesFor both of these purposes it was important that we:Understand the goals of grantees’ SM activitiesIdentify specific metrics of successGo beyond SM metrics to track and identify changes with data we collected “offline”
  • The way that projects were using SM was very diverse. Projects using SM basically fell into two camps:Projects such as the Notebook are community-based news sites. The Notebook is a hyperlocal news site based in Philadelphia that aims to inform parents and educators about what’s happening in education. Most, if not all of their activity is online. Projects such as Grow W-N-Y focus more on engaging citizens. Grow WNY aims to build a network of people to change environmental practices and policies in the city and region. While they have developed an online presence on the web and through SM much of their activity happens offline, via face to face meetings.Our SMM strategies needed to consider that both of these types of projects are part of the Info Challenge.
  • The main takeaway is that comparative metrics are often lifted up as the holy grail of social media measurement (“if only we could compare our # of visitors to their # of visitors…”), but in our case we found that there were so many caveats about the way social media was being used and the types of audiences projects were reaching that it didn’t give us the insights about social  media effectiveness (and not impact). It gave us a directional sense of how projects were doing overall (e.g., 40% of projects were positive on at least one indicator of reach), but we needed other methods to really examine community impact.
  • Measuring Social Media Impact Lessons from Knight Community Information Challenge

    1. 1. Boston | Geneva | Mumbai | San Francisco | Seattle | Washington FSG.ORGPresentation for:Measuring Social Media Impact:A closer look at the Knight Foundation’sCommunity Information ChallengeRWJF Social Media Measurement MeetingPrinceton, NJApril 25, 2013
    2. 2. 2FSG.ORG© 2013 FSGAre communities becoming…More informed?More engaged?Two Questions Guided the Evaluation of theCommunity Information Challenge“I didn’t know how easy itcould be to care for theenvironment.”“I am going to see ifmy neighborhood canstart a recyclingprogram.”
    3. 3. 3FSG.ORG© 2013 FSGWe Utilized the Following Practices inMeasuring Grantees’ Social Media Activities• Understand the goals of socialmedia activity• Identify the metrics of success• Track changes with “offline” dataIMPACT Evaluation Guide
    4. 4. 4FSG.ORG© 2013 FSGUnderstand the Goals of Social Media Activity
    5. 5. 5FSG.ORG© 2013 FSGWe Needed a Framework to Understand the Impactof a Diverse Set of Information Projects
    6. 6. 6FSG.ORG© 2013 FSGOur Social Media Measurement Focused onProject-Level OutcomesReaching target audienceEngaging target audience
    7. 7. 7FSG.ORG© 2013 FSGWe Began Identifying the Metrics of SuccessIncreasing Reach Increasing Engagement• More (new) people visitingthe website over time• More people subscribing toget information updates• More people attendingproject-specific trainings,events, or meetings• More (new) people commentingand contributing informationonline• People taking action online• People sharing information withtheir networks• People participating in trainings,events, or meetingsWe needed to look at changes that would take placeonline and offline
    8. 8. 8FSG.ORG© 2013 FSGThen We Mapped Outcomes to Key PerformanceIndicators (KPIs)Increasing Reach• More (new) people visiting the website over time• More people subscribing to get informationupdates• More people attending project-specific trainings,events, or meetingsLesson Learned: Solicit the help of social media expertsto identify the most relevant social media metricsIncreasing Engagement• More (new) people commenting and contributinginformation online• People taking action online, e.g., submittingcontest entries• People sharing information with their networks% of visits by new visitors over time# of Fans, Followers, registeredusers# of attendees# comments, content uploadsDonations, sign-ups, “conversions”Retweets, forwards, hosting events
    9. 9. 9FSG.ORG© 2013 FSGEntering Data in a Spreadsheet Provided Us withInformation That We Could Use for BenchmarkingTimetrendsGoogleAnalyticsFacebookInsightsTwitter andTweetreachYouTube“Custom”metrics
    10. 10. 10FSG.ORG© 2013 FSGWe Plotted Grantee Data to Better UnderstandProject ReachSource: Anonym zed data from KCIC Year-End Evaluation Report (2010)-3%-2%-1%0%1%2%3%4%5%-2% -1% 0% 1% 2% 3% 4% 5% 6% 7%Average Monthly Growth in VisitsAverageMonthlyGrowthinPercentageofVisitsfromNewVisitorsProjects in this quadrant have shown thegreatest growth in reach through theirwebsite
    11. 11. 11FSG.ORG© 2013 FSGSo What about Impact?
    12. 12. 12FSG.ORG© 2013 FSGTrack Changes with “Offline” DataDoes filling out an online petition or sharing photos through a nonprofit Flickraccount provide information about participation in “real world” activities?OnlineEngagementOfflineEngagementWe needed to examine theintersection of online andoffline behaviors
    13. 13. 13FSG.ORG© 2013 FSGSome Evidence of Real-World EngagementCould Be Found OnlineGrowWNY, Buffalo NYSource photo by Kate Mini Hillman on GrowWNY Flickr photostream (http://www.flickr.com/photos/growwny/6035588130/in/photostream.What actions arepeople taking?
    14. 14. 14FSG.ORG© 2013 FSGWe Collected Data Using More “Traditional”Evaluation Methods to Fully Understand ImpactAdditional Data WereCollected Using… Grantee Surveys Interviews Qualitative (Content) Analysis Audience or Post-Event Surveys Registered User SurveysHow is social mediabeing used? By whom?How are attitudeschanging?Are people actingdifferently?
    15. 15. 15FSG.ORG© 2013 FSGIf online “action” is leading to end-game impact?If we are reaching the already converted?What “good” looks like?Many Questions Still RemainHow Do We Know…

    ×