I’m not a measurement geek, didn’t appreciate it, but was curiousWorked in the nonprofit sector for over 33 years. Had a front row seat at the creation of a field – nonprofit technology – use of technology for mission-driven work. I’m a master trainer so I get to travel around the work and work with changemakers on how to use the tools for social change or mission driven work. Been Visiting Scholar at the David and Lucile Packard Foundation since 2009 – wrote the two Networked Nonprofit Books – design and facilitate peer learning networks on how to integrate social media and measurement.The book on measurement had 60 Packard grantees testing the main framework – KD Paine’s 7 steps of measurement while we were writing book, informed case studies and the practical advice. This year at Packard, the OE program is supporting peer learning groups – using the book to improve measurement practice.-Panel at SXSW in 2009 that had early adopters share their measurement practices – in verse-KD Paine an expert judge – I moderated in Iambic Pente, KD shared advice in heroic couplets – we said, “we should write a book!” -Fast forward 4 years – and the “Measuring the Networked Nonprofit”
Before we dive into measurement, have to talk a little bit about what we’re measuring – transparent organizations that are agile, leverage their networks for social change and masters of using the tools. Not all nonprofits were born this way, had to change. As a capacity builder, learned that adopting new was of working happens incrementally ..
Taken this deeper over the past 7 years to be able to quantify 13 different areas of practice, including measurement and identify whether they are CWRF -Helps organization look at how to intentionally mature their practice – develops a score and look at what to improve -Use as a baseline for peer learning -Helps organizations do “Benchlearning” from one another – imp for networked learning So, if you got a 1.5, it means that you are on your way to walking.https://docs.google.com/spreadsheet/ccc?key=0AtsV5h84LWk0dFhENWFXVzBwZ2lWOGlzazZSek5Iemc#gid=1
Not all nonprofits become data informed, but the practices can be developed incrementally over time. I see nonprofits in all these categories -- and when I do a benchmarking study – I tend to see a bell curve – with more in the Walking and Running stagesLet’s look at some at some more advanced pratitioners
Bob Filbin from DoSomething is here at this meeting – so if you want more details be sure to talk to him today – and he can tweet more detailsDoSomething has a mission to get 1.5 million teens active on social change campaigns by 2015My talk from TED last year sort of summarized the whole thing. (Its only 5 mins.)Basically, it is a help line for kids by text. Terrific to give them support via a mechanism they prefer. Its private (noboy hears you talking.) Blah blahblah.But what makes this a baller idea? The data! We're going to be using natural language tagging (from the MIT Media Lab) to makr key words in real time--and map out youth issues. We'll finally have real data on every youth issue, every zip code, time of day, etc. This information will change EVERYTHING.
http://www.flickr.com/photos/mkrigsman/3428179614/DoSomething has two data analyst positions on staff .. And they aren’t sitting in the corner playing with their spreadsheetsWhile a big part of their job is to become the stewards of the dashboard, they work with staff – so that making sense of data Is not an adhoc process, but one of continous improvement of the programs. The data analysts work collaboratively with staff to help them apply and understand their data.
This is an example of a recent campaign to help reduce the number of dogs/cats being killed in kill sheltersResearch found that this was happening because many aren’t posting good photos on social networks and the internetThey created an app to recruit “furtographers”
Back in the office, the data scientists were looking at the data in real time to figure out what was driving people to their landing page and getting them to sign up.
-Humane Society of US – Carie Lewis participated in the SXSW in 2009 and in 2013 we did the session again at SXSW-HSUS is an early adopter of social media – and in many areas of practice is flying, although they self-assessed their measurement practice at walking to running …-They have org wide KPIs for their social media – customer service wins, fundraising, and actions taken – plus associated metrics that help figure out whether their digital campaigns and social media integration is helping to push the needle on the KPIS
One reason is more cultural than anything else … Data collection is consistent – walking stage, and they are getting to running – getting data to be shared among departments.They have a cross-functional team in place with social media/digital in different departmentsThey have established regular 9 minutes meetings to debrief on campaigns, use a template/common language to collect data.
So, one of their KPIs is donations $ - but if their web department is not going to use source codes – how can they measure conversions from social media traffic referral? In 2011, they didn’t. Through the 9 minute meetings, web team began to understand the importance of doing this – and now can accurately track conversions from social media traffic referral – and make adjustments. They use a report template for all campaigns, all departments – right now a simple word document that summarizes data from excel – from free or low cost tools such as Google Analytics, FB Insights, Twitter Analytics, and Convio. They do an after action review for each campaign, write down what worked and what didn’t and use those reports when planning the next campaign.
Upwell is definitely flying …..Upwell is focused on Ocean Conservation – their goal is to increase engagement and conversation about ocean conversationThey focus on listening, using Radian 6 and monitoring key words like “Ocean” and look at the “chatter” out there on social networks about these topics. They have developed a baseline methodology so they know what the base is for “share of conversation” on a particular ocean conservation conversation they are monitoring. They identify opportunities to engage to “increase the conversation” on the topic and measure it. One might say they have one metric that matters or “Lean impact” - increase conversation about ocean conservation. As a networked nonprofit – and hoping to build a movement, they are also transparent and share practices - iteratehttp://www.bethkanter.org/upwell-campaign/
This chart is probably very appealing to all of you – and if you want details – Rachel is out there hanging out in our hashtag and can point you to more details ….This graphic is a snap shot of their social media monitoring of eight different ocean conservation areas ..Each line represents the social mention volume in one of our issue keyword sets.The pair of pink lines are mentions of sharks and cetaceans. Shark week makes big spikes, but cetaceans get more social mentions a day. Hanging out on the bottom of the graph are tuna, overfishing, the Gulf of mexico, ocean acidification, sustainable seafood and tiny tiny MPAs.
They were monitoring and saw this opportunity around the keyword “Shark” because of shark week was the biggest attention spike within the eight ocean issues they monitor – it provided a big opportunity to expand audiences, and to grow their distributed network. They campaign was to set an ambitious goal to spike a conversation around shark conversation.More detail on Shark Week:We tried about a dozen things to make that Shark Week conversation bump, including live tweeting ourselves, image macros, and a toolkit for shark evangelists. What really worked was the pair of sharkinars we hosted for shark evangelists. We shared that the big attention spike was coming up, reviewed top hashtags, identified shark influencers, and that YAY chart was our tone/sentiment analysis of #sharkweek tweets. We taught the shark evangelists that it was mainly a FAN conversation on Twitter, not at FEAR (sensationalized) conversation.
They packaged content and worked with partners working on Shark – and were able to analyze and attribute the increase of conversation in 2012 to the campaign tactics. Through this, were able to target new activists/champions – not on their Radar through social media conversations.Then that bar chart with the knitting: the shark week conversation grew 109% over 2011, but the shark conservation mentions grew 210% year over year. That made us happy!
We’ve seen some of the flyers but not all nonprofits are quite there – and there is a lot of room for improvement – so I’m going to end with a couple of quick points of about where ….Measurement DisciplineSmall pilots Data LiteracyImproving and Proving Results
-What’s hard is having the conversation to connect social media strategies/metrics to ladder them to organizational outcomes-Takes some time to get everyone on the same page, but can be done …..
In my own work withpackard grantees and the peer learning programs, we have them start with that conversation – and some generic examples of KPIs – most frequent question I got while writing the book and now talking about it – “What is a KPI?”
A lot of shiny object syndrome with metrics tools – big mistake is that they start with the tools – often free analytics – and that gets in the way of good measurement discipline or laddering up to big outcomes ….http://www.flickr.com/photos/leeontheroad/89666692/sizes/z/in/photostream/
The best tool is excelMany of these professional and free tools export to excel – need to do a little more work to make it a dashboard, not just the data – otherwise it is just triva
There’s a big need for improved data literacy – both in-house for the basics – and but also working with measurement and analytics geeks – data nerds, and data scientists. There are a number of orgs that do volunteer matching. Raise the level of in-house literacy – much more effective working with professionals. Peer learning networks are also another good intervention.
http://www.flickr.com/photos/40991157@N02/3923081100/Learning from failure ..
Nonprofit Social Media Measurement: Maturity of PracticeBeth Kanter, Master Trainer, Author, and BloggerRobert Wood Johnson Foundation Convening,April 25, 2013
Social Media Nonprofit ROI Poetry Slam 2009Context
Maturity of Practice Framework: Networked NonprofitsIf you can’t fly then run, if you can’t run then walk, ifyou can’t walk then crawl, but whatever you do youhave to keep moving forward.”
CRAWL WALK RUN FLYLinking Social withResults and NetworksPilot: Focus oneprogram or campaignIncremental CapacityBecomingTransparentLadder ofEngagementContent StrategyBest PracticesUses social mediametrics to learnCommunicationsStrategyDevelopmentCulture ChangeNetwork BuildingFree AgentsMulti-Channel Engagement,Content, and MeasurementKPIs, Reflection andContinuous Improvement
Maturity of Practice: Crawl-Walk-Run-FlyCategories Practices AverageCULTURE Networked Mindset 2.3Institutional Support 1.5CAPACITY Staffing 1.8Strategy 1.5MEASUREMENT Analysis 1.5Tools 2.0Sense-Making 1.8LISTENING Brand Monitoring 1.5Influencer Research 1.3ENGAGEMENT Ladder of Engagement 1.5CONTENT Integration/Optimization 1.8NETWORK Influencer Engagement 2.0Relationship Mapping 1.31 2 3 40.0 0.5 1.0 1.5 2.0 2.5Arts & Cultural Alliance ofCentral FloridaLA Stage AllianceThe Alliance of ResidentTheatres/New YorkAustin Creative AllianceTheatre Bay AreaC4 AtlantaAll Indicators
Crawl Walk Run Flyinconsistent datacollectionData collectionconsistent but notsharedData from multiplesourcesOrg Wide KPIsHaphazard reporting Reporting moreconsistent, but notalways linked tooutcomesSystem and structure fordata collectionOrganizationalDashboard withdifferent views, sharingDecisions based on gut Rarely makes decisionsbased on dataDiscussed at staffmeetings, decisionsmade using itData visualization, real-time reporting, formalreflection processCWRF: Becoming Data Informed: What Does Practice looklike?AnalysisToolsSense-Making
Data-Informed Culture: It starts from the top!Do Something.org
Tear down those silos and walls around data …More time think about that the data, then collect it
HSUS Analysis Process:• Everyone gathers their data• We sit in a room and analyze it• The project manager develops a report of“what we did/what we learned/what werecommend for next time “+ dataChallenges:• Reports aren’t distributed org-wide yet• Still making some decisions based on whatwe think, not supported by data• Upper management still not on board, justwants things to get doneScaling Data-Informed
Jogging!2011: Not Using Source Codes forAll Campaigns To Measure SocialMedia Conversion2013: Using Source Codes for AllCampaigns To Measure SocialMedia Conversion
Pick The Right Data Point(s) To Track Outcomes
Outcomes KPIIncrease donations % reduction in cost per dollar raisedIncrease donor base % increase in new donorsIncrease number of volunteers % increase in volunteersIncrease awareness or dissemination % increase in awareness,% increase in visibility/prominence% increase conversion of downloadsImprove relationships with existingdonors/volunteers% improvement in relationship scores,% increase in donation from existingdonorsImprove engagement withstakeholders% increase in engagement (commentson YouTube, shares on Facebook,comments on blog, etc.Change in behavior % decrease in bad behavior,% increase in good behaviorChange in attitude about yourorganization% increase in trust score , influence,transparency, or relationship scoreIncrease in skills and knowledge ofstaff learningIncrease in revenue per employee,% employees understanding theirroles and organizational mission
• Sentiment• Themes• MessagingContentAnalysis• Attitudes• Preferences• BehaviorSurveyResearch• Reach• Engagement• ActionAnalyticsPick the Right Tool for the Job
Data Literacy: Collect, Clean, Analyze, Visualize, Apply
Step 7 – Analyze ResultsJoyful Funerals Metrics MondaysReflection and Improvement: Learning from Failure
1. Measurement discipline but not at the expense ofbeing networked – balance serendipity withstrategy2. Different stages of maturity, requires incrementalsteps to improve practices3. Linking social media to outcomes requires silobusting for both effective strategy and metrics4. Data literacy - working with experts andorganizationally5. Embrace and learn from failureThe State of Nonprofit Social Media Measurement
Thank you!www.bethkanter.orgwww.facebook.com/beth.kanter.blog@kanter on Twitter