Uploaded byBeth Kanter
RWJF Advancing Social Media Metrics Meeting
This document discusses the maturity of nonprofit social media measurement practices. It presents a framework that nonprofits can follow from "crawl-walk-run-fly" as they progress in their social media measurement. The "crawl" stage involves piloting measurement of one program, while "fly" involves measuring multiple channels, developing key performance indicators (KPIs), and using data for continuous improvement. The document provides examples of nonprofits at different stages and outlines best practices such as linking social media to outcomes, developing a data-informed culture, and embracing failure.
More Related Content
Similar to RWJF Advancing Social Media Metrics Meeting
RWJF Advancing Social Media Metrics Meeting
- 1. Nonprofit Social MediaMeasurement: Maturity of Practice Beth Kanter, Master Trainer, Author, and Blogger Robert Wood Johnson Foundation Convening, April 25, 2013
- 2. Social Media NonprofitROI Poetry Slam 2009 Context
- 3. Maturity of PracticeFramework: Networked Nonprofits If you can’t fly then run, if you can’t run then walk, if you can’t walk then crawl, but whatever you do you have to keep moving forward.”
- 4. CRAWL WALK RUNFLY Linking Social with Results and Networks Pilot: Focus one program or campaign Incremental Capacity Becoming Transparent Ladder of Engagement Content Strategy Best Practices Uses social media metrics to learn Communications Strategy Development Culture Change Network Building Free Agents Multi-Channel Engagement, Content, and Measurement KPIs, Reflection and Continuous Improvement
- 5. Maturity of Practice:Crawl-Walk-Run-Fly Categories Practices Average CULTURE Networked Mindset 2.3 Institutional Support 1.5 CAPACITY Staffing 1.8 Strategy 1.5 MEASUREMENT Analysis 1.5 Tools 2.0 Sense-Making 1.8 LISTENING Brand Monitoring 1.5 Influencer Research 1.3 ENGAGEMENT Ladder of Engagement 1.5 CONTENT Integration/Optimization 1.8 NETWORK Influencer Engagement 2.0 Relationship Mapping 1.3 1 2 3 4 0.0 0.5 1.0 1.5 2.0 2.5 Arts & Cultural Alliance of Central Florida LA Stage Alliance The Alliance of Resident Theatres/New York Austin Creative Alliance Theatre Bay Area C4 Atlanta All Indicators
- 6. The Data InformedNonprofit
- 7. Crawl Walk RunFly inconsistent data collection Data collection consistent but not shared Data from multiple sources Org Wide KPIs Haphazard reporting Reporting more consistent, but not always linked to outcomes System and structure for data collection Organizational Dashboard with different views, sharing Decisions based on gut Rarely makes decisions based on data Discussed at staff meetings, decisions made using it Data visualization, real- time reporting, formal reflection process CWRF: Becoming Data Informed: What Does Practice look like? Analysis Tools Sense-Making
- 8. Data-Informed Culture: Itstarts from the top! Do Something.org
- 9. Tear down thosesilos and walls around data … More time think about that the data, then collect it
- 11.
- 14. HSUS Analysis Process: •Everyone gathers their data • We sit in a room and analyze it • The project manager develops a report of “what we did/what we learned/what we recommend for next time “+ data Challenges: • Reports aren’t distributed org-wide yet • Still making some decisions based on what we think, not supported by data • Upper management still not on board, just wants things to get done Scaling Data-Informed
- 15. Jogging! 2011: Not UsingSource Codes for All Campaigns To Measure Social Media Conversion 2013: Using Source Codes for All Campaigns To Measure Social Media Conversion
- 17.
- 18.
- 19. 133,167 745,030 1,683,670 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 1,600,000 1,800,000 Shark Week 2010Shark Week 2011 Shark Week 2012 One Metric That Matters: Increase Shark Conservation Conversation
- 20. Nonprofit Social MediaMeasurement Practices: Room for Improvement
- 21.
- 22. Pick The RightData Point(s) To Track Outcomes
- 23. Outcomes KPI Increase donations% reduction in cost per dollar raised Increase donor base % increase in new donors Increase number of volunteers % increase in volunteers Increase awareness or dissemination % increase in awareness, % increase in visibility/prominence % increase conversion of downloads Improve relationships with existing donors/volunteers % improvement in relationship scores, % increase in donation from existing donors Improve engagement with stakeholders % increase in engagement (comments on YouTube, shares on Facebook, comments on blog, etc. Change in behavior % decrease in bad behavior, % increase in good behavior Change in attitude about your organization % increase in trust score , influence, transparency, or relationship score Increase in skills and knowledge of staff learning Increase in revenue per employee, % employees understanding their roles and organizational mission
- 24. • Sentiment • Themes •Messaging Content Analysis • Attitudes • Preferences • Behavior Survey Research • Reach • Engagement • Action Analytics Pick the Right Tool for the Job
- 25. Now That’s ASpreadsheet!
- 26. Data Literacy: Collect,Clean, Analyze, Visualize, Apply
- 27. Step 7 –Analyze Results Joyful Funerals Metrics Mondays Reflection and Improvement: Learning from Failure
- 28. 1. Measurement disciplinebut not at the expense of being networked – balance serendipity with strategy 2. Different stages of maturity, requires incremental steps to improve practices 3. Linking social media to outcomes requires silo busting for both effective strategy and metrics 4. Data literacy - working with experts and organizationally 5. Embrace and learn from failure The State of Nonprofit Social Media Measurement
- 29.
Editor's Notes
- #3 I’m not a measurement geek, didn’t appreciate it, but was curiousWorked in the nonprofit sector for over 33 years. Had a front row seat at the creation of a field – nonprofit technology – use of technology for mission-driven work. I’m a master trainer so I get to travel around the work and work with changemakers on how to use the tools for social change or mission driven work. Been Visiting Scholar at the David and Lucile Packard Foundation since 2009 – wrote the two Networked Nonprofit Books – design and facilitate peer learning networks on how to integrate social media and measurement.The book on measurement had 60 Packard grantees testing the main framework – KD Paine’s 7 steps of measurement while we were writing book, informed case studies and the practical advice. This year at Packard, the OE program is supporting peer learning groups – using the book to improve measurement practice.-Panel at SXSW in 2009 that had early adopters share their measurement practices – in verse-KD Paine an expert judge – I moderated in Iambic Pente, KD shared advice in heroic couplets – we said, “we should write a book!” -Fast forward 4 years – and the “Measuring the Networked Nonprofit”
- #4 Before we dive into measurement, have to talk a little bit about what we’re measuring – transparent organizations that are agile, leverage their networks for social change and masters of using the tools. Not all nonprofits were born this way, had to change. As a capacity builder, learned that adopting new was of working happens incrementally ..
- #5 http://www.flickr.com/photos/oreoqueen/3235090633/in/faves-cambodia4kidsorg/http://www.flickr.com/photos/jonathandesign/7031920221/in/faves-cambodia4kidsorg/http://www.flickr.com/photos/bdfbrasil/2416260064/sizes/m/in/faves-cambodia4kidsorg/http://www.flickr.com/photos/levymh/6891554365/in/faves-cambodia4kidsorg/
- #6 Taken this deeper over the past 7 years to be able to quantify 13 different areas of practice, including measurement and identify whether they are CWRF -Helps organization look at how to intentionally mature their practice – develops a score and look at what to improve -Use as a baseline for peer learning -Helps organizations do “Benchlearning” from one another – imp for networked learning So, if you got a 1.5, it means that you are on your way to walking.https://docs.google.com/spreadsheet/ccc?key=0AtsV5h84LWk0dFhENWFXVzBwZ2lWOGlzazZSek5Iemc#gid=1
- #8 Not all nonprofits become data informed, but the practices can be developed incrementally over time. I see nonprofits in all these categories -- and when I do a benchmarking study – I tend to see a bell curve – with more in the Walking and Running stagesLet’s look at some at some more advanced pratitioners
- #9 Bob Filbin from DoSomething is here at this meeting – so if you want more details be sure to talk to him today – and he can tweet more detailsDoSomething has a mission to get 1.5 million teens active on social change campaigns by 2015My talk from TED last year sort of summarized the whole thing. (Its only 5 mins.)Basically, it is a help line for kids by text. Terrific to give them support via a mechanism they prefer. Its private (noboy hears you talking.) Blah blahblah.But what makes this a baller idea? The data! We're going to be using natural language tagging (from the MIT Media Lab) to makr key words in real time--and map out youth issues. We'll finally have real data on every youth issue, every zip code, time of day, etc. This information will change EVERYTHING.
- #10 http://www.flickr.com/photos/mkrigsman/3428179614/DoSomething has two data analyst positions on staff .. And they aren’t sitting in the corner playing with their spreadsheetsWhile a big part of their job is to become the stewards of the dashboard, they work with staff – so that making sense of data Is not an adhoc process, but one of continous improvement of the programs. The data analysts work collaboratively with staff to help them apply and understand their data.
- #11 This is an example of a recent campaign to help reduce the number of dogs/cats being killed in kill sheltersResearch found that this was happening because many aren’t posting good photos on social networks and the internetThey created an app to recruit “furtographers”
- #12 LauncheNo addhttp://www.youtube.com/watch?v=QhgsJjNVbu0http://gawker.com/5950941/kathie-lee-dropped-a-puppy-on-his-head-on-live-tv-todayhttp://www.youtube.com/watch?v=sQzo_3yIc8M
- #13 Back in the office, the data scientists were looking at the data in real time to figure out what was driving people to their landing page and getting them to sign up.
- #14 -Humane Society of US – Carie Lewis participated in the SXSW in 2009 and in 2013 we did the session again at SXSW-HSUS is an early adopter of social media – and in many areas of practice is flying, although they self-assessed their measurement practice at walking to running …-They have org wide KPIs for their social media – customer service wins, fundraising, and actions taken – plus associated metrics that help figure out whether their digital campaigns and social media integration is helping to push the needle on the KPIS
- #15 One reason is more cultural than anything else … Data collection is consistent – walking stage, and they are getting to running – getting data to be shared among departments.They have a cross-functional team in place with social media/digital in different departmentsThey have established regular 9 minutes meetings to debrief on campaigns, use a template/common language to collect data.
- #16 So, one of their KPIs is donations $ - but if their web department is not going to use source codes – how can they measure conversions from social media traffic referral? In 2011, they didn’t. Through the 9 minute meetings, web team began to understand the importance of doing this – and now can accurately track conversions from social media traffic referral – and make adjustments. They use a report template for all campaigns, all departments – right now a simple word document that summarizes data from excel – from free or low cost tools such as Google Analytics, FB Insights, Twitter Analytics, and Convio. They do an after action review for each campaign, write down what worked and what didn’t and use those reports when planning the next campaign.
- #17 Upwell is definitely flying …..Upwell is focused on Ocean Conservation – their goal is to increase engagement and conversation about ocean conversationThey focus on listening, using Radian 6 and monitoring key words like “Ocean” and look at the “chatter” out there on social networks about these topics. They have developed a baseline methodology so they know what the base is for “share of conversation” on a particular ocean conservation conversation they are monitoring. They identify opportunities to engage to “increase the conversation” on the topic and measure it. One might say they have one metric that matters or “Lean impact” - increase conversation about ocean conservation. As a networked nonprofit – and hoping to build a movement, they are also transparent and share practices - iteratehttp://www.bethkanter.org/upwell-campaign/
- #18 This chart is probably very appealing to all of you – and if you want details – Rachel is out there hanging out in our hashtag and can point you to more details ….This graphic is a snap shot of their social media monitoring of eight different ocean conservation areas ..Each line represents the social mention volume in one of our issue keyword sets.The pair of pink lines are mentions of sharks and cetaceans. Shark week makes big spikes, but cetaceans get more social mentions a day. Hanging out on the bottom of the graph are tuna, overfishing, the Gulf of mexico, ocean acidification, sustainable seafood and tiny tiny MPAs.
- #19 They were monitoring and saw this opportunity around the keyword “Shark” because of shark week was the biggest attention spike within the eight ocean issues they monitor – it provided a big opportunity to expand audiences, and to grow their distributed network. They campaign was to set an ambitious goal to spike a conversation around shark conversation.More detail on Shark Week:We tried about a dozen things to make that Shark Week conversation bump, including live tweeting ourselves, image macros, and a toolkit for shark evangelists. What really worked was the pair of sharkinars we hosted for shark evangelists. We shared that the big attention spike was coming up, reviewed top hashtags, identified shark influencers, and that YAY chart was our tone/sentiment analysis of #sharkweek tweets. We taught the shark evangelists that it was mainly a FAN conversation on Twitter, not at FEAR (sensationalized) conversation.
- #20 They packaged content and worked with partners working on Shark – and were able to analyze and attribute the increase of conversation in 2012 to the campaign tactics. Through this, were able to target new activists/champions – not on their Radar through social media conversations.Then that bar chart with the knitting: the shark week conversation grew 109% over 2011, but the shark conservation mentions grew 210% year over year. That made us happy!
- #21 We’ve seen some of the flyers but not all nonprofits are quite there – and there is a lot of room for improvement – so I’m going to end with a couple of quick points of about where ….Measurement DisciplineSmall pilots Data LiteracyImproving and Proving Results
- #23 -What’s hard is having the conversation to connect social media strategies/metrics to ladder them to organizational outcomes-Takes some time to get everyone on the same page, but can be done …..
- #24 In my own work withpackard grantees and the peer learning programs, we have them start with that conversation – and some generic examples of KPIs – most frequent question I got while writing the book and now talking about it – “What is a KPI?”
- #25 A lot of shiny object syndrome with metrics tools – big mistake is that they start with the tools – often free analytics – and that gets in the way of good measurement discipline or laddering up to big outcomes ….http://www.flickr.com/photos/leeontheroad/89666692/sizes/z/in/photostream/
- #26 The best tool is excelMany of these professional and free tools export to excel – need to do a little more work to make it a dashboard, not just the data – otherwise it is just triva
- #27 There’s a big need for improved data literacy – both in-house for the basics – and but also working with measurement and analytics geeks – data nerds, and data scientists. There are a number of orgs that do volunteer matching. Raise the level of in-house literacy – much more effective working with professionals. Peer learning networks are also another good intervention.
- #28 http://www.flickr.com/photos/40991157@N02/3923081100/Learning from failure ..
- #30 http://bit.ly/network-leadership