Measuring Social Media Impact Lessons from Knight Community Information Challenge
•Download as PPTX, PDF•
4 likes•5,149 views
This document discusses FSG's approach to measuring the social media impact of the Knight Foundation's Community Information Challenge grants. It outlines the key questions and goals that guided the evaluation, including whether communities became more informed or engaged. FSG identified metrics like reach and engagement to measure grantees' social media activities. They tracked online metrics over time and also used traditional evaluation methods to understand how online activities related to offline impacts, like changes in attitudes or behaviors. While some evidence of real-world engagement was found online, many questions still remained about linking online and offline impacts.
Recommended
Recommended
More Related Content
What's hot
What's hot (19)
Similar to Measuring Social Media Impact Lessons from Knight Community Information Challenge
Similar to Measuring Social Media Impact Lessons from Knight Community Information Challenge (20)
Recently uploaded
Recently uploaded (20)
Measuring Social Media Impact Lessons from Knight Community Information Challenge
- 1. Boston | Geneva | Mumbai | San Francisco | Seattle | Washington FSG.ORG Presentation for: Measuring Social Media Impact: A closer look at the Knight Foundation’s Community Information Challenge RWJF Social Media Measurement Meeting Princeton, NJ April 25, 2013
- 2. 2 FSG.ORG © 2013 FSG Are communities becoming… More informed? More engaged? Two Questions Guided the Evaluation of the Community Information Challenge “I didn’t know how easy it could be to care for the environment.” “I am going to see if my neighborhood can start a recycling program.”
- 3. 3 FSG.ORG © 2013 FSG We Utilized the Following Practices in Measuring Grantees’ Social Media Activities • Understand the goals of social media activity • Identify the metrics of success • Track changes with “offline” data IMPACT Evaluation Guide
- 4. 4 FSG.ORG © 2013 FSG Understand the Goals of Social Media Activity
- 5. 5 FSG.ORG © 2013 FSG We Needed a Framework to Understand the Impact of a Diverse Set of Information Projects
- 6. 6 FSG.ORG © 2013 FSG Our Social Media Measurement Focused on Project-Level Outcomes Reaching target audience Engaging target audience
- 7. 7 FSG.ORG © 2013 FSG We Began Identifying the Metrics of Success Increasing Reach Increasing Engagement • More (new) people visiting the website over time • More people subscribing to get information updates • More people attending project-specific trainings, events, or meetings • More (new) people commenting and contributing information online • People taking action online • People sharing information with their networks • People participating in trainings, events, or meetings We needed to look at changes that would take place online and offline
- 8. 8 FSG.ORG © 2013 FSG Then We Mapped Outcomes to Key Performance Indicators (KPIs) Increasing Reach • More (new) people visiting the website over time • More people subscribing to get information updates • More people attending project-specific trainings, events, or meetings Lesson Learned: Solicit the help of social media experts to identify the most relevant social media metrics Increasing Engagement • More (new) people commenting and contributing information online • People taking action online, e.g., submitting contest entries • People sharing information with their networks % of visits by new visitors over time # of Fans, Followers, registered users # of attendees # comments, content uploads Donations, sign-ups, “conversions” Retweets, forwards, hosting events
- 9. 9 FSG.ORG © 2013 FSG Entering Data in a Spreadsheet Provided Us with Information That We Could Use for Benchmarking Time trends Google Analytics Facebook Insights Twitter and Tweetreach YouTube “Custom” metrics
- 10. 10 FSG.ORG © 2013 FSG We Plotted Grantee Data to Better Understand Project Reach Source: Anonym zed data from KCIC Year-End Evaluation Report (2010) -3% -2% -1% 0% 1% 2% 3% 4% 5% -2% -1% 0% 1% 2% 3% 4% 5% 6% 7% Average Monthly Growth in Visits AverageMonthlyGrowthinPercentageof VisitsfromNewVisitors Projects in this quadrant have shown the greatest growth in reach through their website
- 11. 11 FSG.ORG © 2013 FSG So What about Impact?
- 12. 12 FSG.ORG © 2013 FSG Track Changes with “Offline” Data Does filling out an online petition or sharing photos through a nonprofit Flickr account provide information about participation in “real world” activities? Online Engagement Offline Engagement We needed to examine the intersection of online and offline behaviors
- 13. 13 FSG.ORG © 2013 FSG Some Evidence of Real-World Engagement Could Be Found Online GrowWNY, Buffalo NY Source photo by Kate Mini Hillman on GrowWNY Flickr photostream (http://www.flickr.com/photos/growwny/6035588130/in/photostream. What actions are people taking?
- 14. 14 FSG.ORG © 2013 FSG We Collected Data Using More “Traditional” Evaluation Methods to Fully Understand Impact Additional Data Were Collected Using… Grantee Surveys Interviews Qualitative (Content) Analysis Audience or Post-Event Surveys Registered User Surveys How is social media being used? By whom? How are attitudes changing? Are people acting differently?
- 15. 15 FSG.ORG © 2013 FSG If online “action” is leading to end-game impact? If we are reaching the already converted? What “good” looks like? Many Questions Still Remain How Do We Know…
Editor's Notes
- Our evaluation examined whether Knight funded grantees were contributing to more INFORMED and ENAGAGED communities. Our evaluation went far beyond SMM We conducted an annual grantee surveyInterviewed project and foundation staffConducted a few case studiesSocial media was one tool that we being used by projects to achieve their goals
- Some of our key learnings around evaluating information and media projects is found in a publication we entitled IMPACT – which can be found on FSG or Knight’s website.Our SMM activities were used for two primary purposes:To examine the effectiveness of SM strategies across granteesTo look at the impact of SM strategies within particular projects, through more in-depth case studiesFor both of these purposes it was important that we:Understand the goals of grantees’ SM activitiesIdentify specific metrics of successGo beyond SM metrics to track and identify changes with data we collected “offline”
- The way that projects were using SM was very diverse. Projects using SM basically fell into two camps:Projects such as the Notebook are community-based news sites. The Notebook is a hyperlocal news site based in Philadelphia that aims to inform parents and educators about what’s happening in education. Most, if not all of their activity is online. Projects such as Grow W-N-Y focus more on engaging citizens. Grow WNY aims to build a network of people to change environmental practices and policies in the city and region. While they have developed an online presence on the web and through SM much of their activity happens offline, via face to face meetings.Our SMM strategies needed to consider that both of these types of projects are part of the Info Challenge.
- The main takeaway is that comparative metrics are often lifted up as the holy grail of social media measurement (“if only we could compare our # of visitors to their # of visitors…”), but in our case we found that there were so many caveats about the way social media was being used and the types of audiences projects were reaching that it didn’t give us the insights about social media effectiveness (and not impact). It gave us a directional sense of how projects were doing overall (e.g., 40% of projects were positive on at least one indicator of reach), but we needed other methods to really examine community impact.