Our evaluation examined whether Knight funded grantees were contributing to more INFORMED and ENAGAGED communities. Our evaluation went far beyond SMM We conducted an annual grantee surveyInterviewed project and foundation staffConducted a few case studiesSocial media was one tool that we being used by projects to achieve their goals
Some of our key learnings around evaluating information and media projects is found in a publication we entitled IMPACT – which can be found on FSG or Knight’s website.Our SMM activities were used for two primary purposes:To examine the effectiveness of SM strategies across granteesTo look at the impact of SM strategies within particular projects, through more in-depth case studiesFor both of these purposes it was important that we:Understand the goals of grantees’ SM activitiesIdentify specific metrics of successGo beyond SM metrics to track and identify changes with data we collected “offline”
The way that projects were using SM was very diverse. Projects using SM basically fell into two camps:Projects such as the Notebook are community-based news sites. The Notebook is a hyperlocal news site based in Philadelphia that aims to inform parents and educators about what’s happening in education. Most, if not all of their activity is online. Projects such as Grow W-N-Y focus more on engaging citizens. Grow WNY aims to build a network of people to change environmental practices and policies in the city and region. While they have developed an online presence on the web and through SM much of their activity happens offline, via face to face meetings.Our SMM strategies needed to consider that both of these types of projects are part of the Info Challenge.
The main takeaway is that comparative metrics are often lifted up as the holy grail of social media measurement (“if only we could compare our # of visitors to their # of visitors…”), but in our case we found that there were so many caveats about the way social media was being used and the types of audiences projects were reaching that it didn’t give us the insights about social media effectiveness (and not impact). It gave us a directional sense of how projects were doing overall (e.g., 40% of projects were positive on at least one indicator of reach), but we needed other methods to really examine community impact.
Measuring Social Media Impact Lessons from Knight Community Information Challenge
Boston | Geneva | Mumbai | San Francisco | Seattle | Washington FSG.ORGPresentation for:Measuring Social Media Impact:A closer look at the Knight Foundation’sCommunity Information ChallengeRWJF Social Media Measurement MeetingPrinceton, NJApril 25, 2013