1. Nonprofit Social Media Measurement: Maturity of Practice
Beth Kanter, Master Trainer, Author, and Blogger
Robert Wood Johnson Foundation Convening,
April 25, 2013
3. Maturity of Practice Framework: Networked Nonprofits
If you can’t fly then run, if you can’t run then walk, if
you can’t walk then crawl, but whatever you do you
have to keep moving forward.”
4. CRAWL WALK RUN FLY
Linking Social with
Results and Networks
Pilot: Focus one
program or campaign
Incremental Capacity
Becoming
Transparent
Ladder of
Engagement
Content Strategy
Best Practices
Uses social media
metrics to learn
Communications
Strategy
Development
Culture Change
Network Building
Free Agents
Multi-Channel Engagement,
Content, and Measurement
KPIs, Reflection and
Continuous Improvement
5. Maturity of Practice: Crawl-Walk-Run-Fly
Categories Practices Average
CULTURE Networked Mindset 2.3
Institutional Support 1.5
CAPACITY Staffing 1.8
Strategy 1.5
MEASUREMENT Analysis 1.5
Tools 2.0
Sense-Making 1.8
LISTENING Brand Monitoring 1.5
Influencer Research 1.3
ENGAGEMENT Ladder of Engagement 1.5
CONTENT Integration/Optimization 1.8
NETWORK Influencer Engagement 2.0
Relationship Mapping 1.3
1 2 3 4
0.0 0.5 1.0 1.5 2.0 2.5
Arts & Cultural Alliance of
Central Florida
LA Stage Alliance
The Alliance of Resident
Theatres/New York
Austin Creative Alliance
Theatre Bay Area
C4 Atlanta
All Indicators
7. Crawl Walk Run Fly
inconsistent data
collection
Data collection
consistent but not
shared
Data from multiple
sources
Org Wide KPIs
Haphazard reporting Reporting more
consistent, but not
always linked to
outcomes
System and structure for
data collection
Organizational
Dashboard with
different views, sharing
Decisions based on gut Rarely makes decisions
based on data
Discussed at staff
meetings, decisions
made using it
Data visualization, real-
time reporting, formal
reflection process
CWRF: Becoming Data Informed: What Does Practice look
like?
Analysis
Tools
Sense-Making
14. HSUS Analysis Process:
• Everyone gathers their data
• We sit in a room and analyze it
• The project manager develops a report of
“what we did/what we learned/what we
recommend for next time “+ data
Challenges:
• Reports aren’t distributed org-wide yet
• Still making some decisions based on what
we think, not supported by data
• Upper management still not on board, just
wants things to get done
Scaling Data-Informed
15. Jogging!
2011: Not Using Source Codes for
All Campaigns To Measure Social
Media Conversion
2013: Using Source Codes for All
Campaigns To Measure Social
Media Conversion
23. Outcomes KPI
Increase donations % reduction in cost per dollar raised
Increase donor base % increase in new donors
Increase number of volunteers % increase in volunteers
Increase awareness or dissemination % increase in awareness,
% increase in visibility/prominence
% increase conversion of downloads
Improve relationships with existing
donors/volunteers
% improvement in relationship scores,
% increase in donation from existing
donors
Improve engagement with
stakeholders
% increase in engagement (comments
on YouTube, shares on Facebook,
comments on blog, etc.
Change in behavior % decrease in bad behavior,
% increase in good behavior
Change in attitude about your
organization
% increase in trust score , influence,
transparency, or relationship score
Increase in skills and knowledge of
staff learning
Increase in revenue per employee,
% employees understanding their
roles and organizational mission
24. • Sentiment
• Themes
• Messaging
Content
Analysis
• Attitudes
• Preferences
• Behavior
Survey
Research
• Reach
• Engagement
• Action
Analytics
Pick the Right Tool for the Job
27. Step 7 – Analyze Results
Joyful Funerals Metrics Mondays
Reflection and Improvement: Learning from Failure
28. 1. Measurement discipline but not at the expense of
being networked – balance serendipity with
strategy
2. Different stages of maturity, requires incremental
steps to improve practices
3. Linking social media to outcomes requires silo
busting for both effective strategy and metrics
4. Data literacy - working with experts and
organizationally
5. Embrace and learn from failure
The State of Nonprofit Social Media Measurement
I’m not a measurement geek, didn’t appreciate it, but was curiousWorked in the nonprofit sector for over 33 years. Had a front row seat at the creation of a field – nonprofit technology – use of technology for mission-driven work. I’m a master trainer so I get to travel around the work and work with changemakers on how to use the tools for social change or mission driven work. Been Visiting Scholar at the David and Lucile Packard Foundation since 2009 – wrote the two Networked Nonprofit Books – design and facilitate peer learning networks on how to integrate social media and measurement.The book on measurement had 60 Packard grantees testing the main framework – KD Paine’s 7 steps of measurement while we were writing book, informed case studies and the practical advice. This year at Packard, the OE program is supporting peer learning groups – using the book to improve measurement practice.-Panel at SXSW in 2009 that had early adopters share their measurement practices – in verse-KD Paine an expert judge – I moderated in Iambic Pente, KD shared advice in heroic couplets – we said, “we should write a book!” -Fast forward 4 years – and the “Measuring the Networked Nonprofit”
Before we dive into measurement, have to talk a little bit about what we’re measuring – transparent organizations that are agile, leverage their networks for social change and masters of using the tools. Not all nonprofits were born this way, had to change. As a capacity builder, learned that adopting new was of working happens incrementally ..
Taken this deeper over the past 7 years to be able to quantify 13 different areas of practice, including measurement and identify whether they are CWRF -Helps organization look at how to intentionally mature their practice – develops a score and look at what to improve -Use as a baseline for peer learning -Helps organizations do “Benchlearning” from one another – imp for networked learning So, if you got a 1.5, it means that you are on your way to walking.https://docs.google.com/spreadsheet/ccc?key=0AtsV5h84LWk0dFhENWFXVzBwZ2lWOGlzazZSek5Iemc#gid=1
Not all nonprofits become data informed, but the practices can be developed incrementally over time. I see nonprofits in all these categories -- and when I do a benchmarking study – I tend to see a bell curve – with more in the Walking and Running stagesLet’s look at some at some more advanced pratitioners
Bob Filbin from DoSomething is here at this meeting – so if you want more details be sure to talk to him today – and he can tweet more detailsDoSomething has a mission to get 1.5 million teens active on social change campaigns by 2015My talk from TED last year sort of summarized the whole thing. (Its only 5 mins.)Basically, it is a help line for kids by text. Terrific to give them support via a mechanism they prefer. Its private (noboy hears you talking.) Blah blahblah.But what makes this a baller idea? The data! We're going to be using natural language tagging (from the MIT Media Lab) to makr key words in real time--and map out youth issues. We'll finally have real data on every youth issue, every zip code, time of day, etc. This information will change EVERYTHING.
http://www.flickr.com/photos/mkrigsman/3428179614/DoSomething has two data analyst positions on staff .. And they aren’t sitting in the corner playing with their spreadsheetsWhile a big part of their job is to become the stewards of the dashboard, they work with staff – so that making sense of data Is not an adhoc process, but one of continous improvement of the programs. The data analysts work collaboratively with staff to help them apply and understand their data.
This is an example of a recent campaign to help reduce the number of dogs/cats being killed in kill sheltersResearch found that this was happening because many aren’t posting good photos on social networks and the internetThey created an app to recruit “furtographers”
Back in the office, the data scientists were looking at the data in real time to figure out what was driving people to their landing page and getting them to sign up.
-Humane Society of US – Carie Lewis participated in the SXSW in 2009 and in 2013 we did the session again at SXSW-HSUS is an early adopter of social media – and in many areas of practice is flying, although they self-assessed their measurement practice at walking to running …-They have org wide KPIs for their social media – customer service wins, fundraising, and actions taken – plus associated metrics that help figure out whether their digital campaigns and social media integration is helping to push the needle on the KPIS
One reason is more cultural than anything else … Data collection is consistent – walking stage, and they are getting to running – getting data to be shared among departments.They have a cross-functional team in place with social media/digital in different departmentsThey have established regular 9 minutes meetings to debrief on campaigns, use a template/common language to collect data.
So, one of their KPIs is donations $ - but if their web department is not going to use source codes – how can they measure conversions from social media traffic referral? In 2011, they didn’t. Through the 9 minute meetings, web team began to understand the importance of doing this – and now can accurately track conversions from social media traffic referral – and make adjustments. They use a report template for all campaigns, all departments – right now a simple word document that summarizes data from excel – from free or low cost tools such as Google Analytics, FB Insights, Twitter Analytics, and Convio. They do an after action review for each campaign, write down what worked and what didn’t and use those reports when planning the next campaign.
Upwell is definitely flying …..Upwell is focused on Ocean Conservation – their goal is to increase engagement and conversation about ocean conversationThey focus on listening, using Radian 6 and monitoring key words like “Ocean” and look at the “chatter” out there on social networks about these topics. They have developed a baseline methodology so they know what the base is for “share of conversation” on a particular ocean conservation conversation they are monitoring. They identify opportunities to engage to “increase the conversation” on the topic and measure it. One might say they have one metric that matters or “Lean impact” - increase conversation about ocean conservation. As a networked nonprofit – and hoping to build a movement, they are also transparent and share practices - iteratehttp://www.bethkanter.org/upwell-campaign/
This chart is probably very appealing to all of you – and if you want details – Rachel is out there hanging out in our hashtag and can point you to more details ….This graphic is a snap shot of their social media monitoring of eight different ocean conservation areas ..Each line represents the social mention volume in one of our issue keyword sets.The pair of pink lines are mentions of sharks and cetaceans. Shark week makes big spikes, but cetaceans get more social mentions a day. Hanging out on the bottom of the graph are tuna, overfishing, the Gulf of mexico, ocean acidification, sustainable seafood and tiny tiny MPAs.
They were monitoring and saw this opportunity around the keyword “Shark” because of shark week was the biggest attention spike within the eight ocean issues they monitor – it provided a big opportunity to expand audiences, and to grow their distributed network. They campaign was to set an ambitious goal to spike a conversation around shark conversation.More detail on Shark Week:We tried about a dozen things to make that Shark Week conversation bump, including live tweeting ourselves, image macros, and a toolkit for shark evangelists. What really worked was the pair of sharkinars we hosted for shark evangelists. We shared that the big attention spike was coming up, reviewed top hashtags, identified shark influencers, and that YAY chart was our tone/sentiment analysis of #sharkweek tweets. We taught the shark evangelists that it was mainly a FAN conversation on Twitter, not at FEAR (sensationalized) conversation.
They packaged content and worked with partners working on Shark – and were able to analyze and attribute the increase of conversation in 2012 to the campaign tactics. Through this, were able to target new activists/champions – not on their Radar through social media conversations.Then that bar chart with the knitting: the shark week conversation grew 109% over 2011, but the shark conservation mentions grew 210% year over year. That made us happy!
We’ve seen some of the flyers but not all nonprofits are quite there – and there is a lot of room for improvement – so I’m going to end with a couple of quick points of about where ….Measurement DisciplineSmall pilots Data LiteracyImproving and Proving Results
-What’s hard is having the conversation to connect social media strategies/metrics to ladder them to organizational outcomes-Takes some time to get everyone on the same page, but can be done …..
In my own work withpackard grantees and the peer learning programs, we have them start with that conversation – and some generic examples of KPIs – most frequent question I got while writing the book and now talking about it – “What is a KPI?”
A lot of shiny object syndrome with metrics tools – big mistake is that they start with the tools – often free analytics – and that gets in the way of good measurement discipline or laddering up to big outcomes ….http://www.flickr.com/photos/leeontheroad/89666692/sizes/z/in/photostream/
The best tool is excelMany of these professional and free tools export to excel – need to do a little more work to make it a dashboard, not just the data – otherwise it is just triva
There’s a big need for improved data literacy – both in-house for the basics – and but also working with measurement and analytics geeks – data nerds, and data scientists. There are a number of orgs that do volunteer matching. Raise the level of in-house literacy – much more effective working with professionals. Peer learning networks are also another good intervention.
http://www.flickr.com/photos/40991157@N02/3923081100/Learning from failure ..
