1. Peer Exchange Group 2: Measuring the Networked Nonprofit: Proving Results To Improve Session 3: September 23, 2011 Why Does Bad Measurement Practice Happen To Good Nonprofits? Beth Kanter, Visiting Scholar, Social Media and Nonprofits The David and Lucile Packard Foundation Organizational Effectiveness Program
2. Welcome! If you experience any technical difficulties logging into the system, please contact Ready/Talk Customer support: 800.843.9166 Please use *6 to Mute your conference line While we are waiting, type into the chat:What is the SMART objective you are measuring or what is your research/learning question?
3. This call is being recorded * 2 Flickr Photo by Malinki
4. Peer Exchange Group 2: Measuring the Networked Nonprofit: Proving Results To Improve Session 3: September 23, 2011 Why Does Bad Measurement Practice Happen To Good Nonprofits? Beth Kanter, Visiting Scholar, Social Media and Nonprofits The David and Lucile Packard Foundation Organizational Effectiveness Program
5. Don’t forget to jot down your reflections in your “wiki journal” On The Call Today Beth Kanter Becky Jain
6. Agenda A Few Reminders Last Month: Quick ReviewBeth’s Reflection/JournalMeasurement Pilot: Peer Share Good and Bad Measurement Practices Next Session Reflection Ask Your Questions and Share Your Insights in the Chat! Hashtag: #measurenp
7. Reminders Wiki JournalsFacebook GroupOffice HoursScoop.It Resources Don’t forget to jot down your reflections in your “wiki journal”
12. Beth’s Journal Program Outcomes At least 50% of 27 participants implement a social media measurement pilot that helps document value/results or improve social media practice by end of the Peer Group 1/2012 At least 50% of 27 participants, generate case studies and/or insightful quotes for the Measuring the Networked Nonprofit book by 1/2012 http://measure-netnon.wikispaces.com/Beth-Kanter-Journal
13. Beth’s Journal 85% have written about their pilot in the wiki journal 44% have had used the office hours 90% have joined FB group 90% attendance on calls (playbacks) http://measure-netnon.wikispaces.com/Beth-Kanter-Journal
14. 85% of the 27 participants have identified a pilot compared to 50% last month
15. Beth’s Observation -Have not started planning or designing project (14%)-Have started planning/designing, but not finished because stuck or have questions (48%) -Have been planning/designing, ready or almost ready to go (28%)
16. CHAT: If you have started planning/designing, but not finished - why? Are you stuck? Do you have questions? What do you need to move forward?
17. Leopold Leadership Target audience: 173 current and past Leopold Leadership Fellows (environmental researchers from a range of science and social science disciplines with a wide range of social media experience)
18. Goals:1) To begin the process of communicating with fellows about how “science leadership 2.0” is different from “science leadership 1.0.”2) To get timely feedback about 3 aspects of our strategic plan from as many of our fellows as possible3) To seed the habit of fellow-to-fellow/group conversation about best practices as an important part of what fellows do as members of our networkWhat failure looks like: Nothing changes; we receive feedback about our 3 questions via individual private communicationWhat success looks like: Fellows engage in a conversation with each other via our listserv about our three questions; new insights and ideas emerge that help us finalize a plan that will provide maximum benefit to our community; the experience of participating in this conversation sparks interest in creating an ongoing space for conversation about best practice.
19. KPIs:Numbers of:Downloads of new strategic plan Replies to director’s message seeking feedback on our listserv vs. replies individually to her or to me Comments on our 3 questions Comments on key messages about how 2.0 is different from 1.0 “Builds” on others’ ideas on the listserv Twosies? Closed triangles? Project ideas? Strategy:We post our strategic plan on our website; our faculty director sends a message to fellows via the listserv seeking feedback on 3 questions and encouraging them to reply to the listserv (rather than privately to her or to me). LinkedIn
20. the goal of GlobalGiving’s unmarketing strategy: To create and share engaging content that informs, inspires, connects and compels - GlobalGiving users, (donors, partners, non-profit project leaders and corporate partners) - potential partners,(potential donors, potential project leaders and potential corporate partners) - and our peers.
22. (some of our) unmarketing objectives: Donor-level: Drive retail donations from current and potential donors Project-level: Demonstrate value to non-profit project leaders and potential project leaders Peer-level: Maintain GG’s thought-leadership role Corporate Partner-level: Demonstrate value to corporate partners and potential project leaders Follower-level: Maintain a consistent, engaging voice which embodies the GG brand & values
23. measurement priorities: Monitoring how well our FB/twitter activities match our social media plan (ratio) Quantifying and qualifying FB/twitter engagement Establishing baselines for website traffic &website conversions from FB/twitter activities (for future measurement!)
24. Key performance metrics Maintain an average daily click rate of 110 clicks on bit.ly links (as measured by Thrive) Post 3-6 engaging messages to Facebook each week; achieve at least 20,000 impressions and 0.1% feedback on each post (as measured by Facebook Insights) Post 20-25 engaging tweets per week; maintain at least an average of 20 RTs per day (as measured by Thrive)
25. What work does this entail? Making sure all of my content is trackable: Posting links using both bit.ly and Google Analytics Campaign urls Posting on Twitter through Thrive Analyzing Thrive Reports and Facebook Insights at on a fortnightly basis Acting on information from reports
26. Why does calculating give us the collywobbles? The 5 Deadly Fears That Get in the Way of Good Measurement
32. CHAT: Anything missing from the list? How to get past this?
33. The 7 Habits of Highly Effective Nonprofits 1. Visualize success and failure 2. Spend more time identifying what you want to measure, not how to measure it 3. Measure in context – don’t ever collect data unless you have SMART objectives 4. Don’t wait until the end of project to collect data 5. Don’t ever just shovel data over the fence and onto your executive director’s desk 6. Less is more
34. Next Session:October 27th11:00 am PST/ 2:00 EST Homework: Complete the description of your project using worksheet from Session 2. (SMART objective, Strategy, Benchmark, and KPI). Identify what you need to move forward – what’s keeping you back Worksheets on Wiki Office Hours
Editor's Notes
Welcome – count down
Remind people to call for tech support*6 to mute conference line*7unmute
http://www.flickr.com/photos/malinki/2621920871/sizes/o/Start recording about 2 minutes late to let people join *2
Welcome – count down
Beth will read list of participantsBeth will introduce herself“The last few calls I realized that it was hard for me to moderate, work the software, take notes, read the chat and present. So, I’m lucky to have recruited a fabulous co-moderator, Becky Jain. She is going to help me with the back channel – the chat.Becky is a blogger and active on many social media communities. After studying at Wesleyan and the London School of Economics, she came to India to live with her husband, and to work in the development sector. Say hello!
This is our agenda – we’ll pause along the way for questions.Use this #measurenp on Twitter
If you haven’t started keeping your wiki journal, please try to spend a few minutes on the site.If you haven’t joined the Facebook Group, please do so https://www.facebook.com/groups/193416010707686?ap=1Remember, office hours if you’re feeling stuck. I’ve added some more optionshttp://measure-netnon.wikispaces.com/Office+Hours_Group+2As soon as we get the pilot projects organized, I’ll add some small group cohort coaching time
Review Steps for Planning Pilot1. Define Results: SMART objective/audience, research needed?2. Identify your strategy, identify the costs, both in direct costs, allocation of resources, and long term costs3. Agree on benchmarks 4. Define your KPI5. Pick the right measurement tool6. Figure out what the data means, change and measure again
In KD Paine’s decades of experience, the measurement process looks like this – no matter what you’re measuring to get at measuring what matters … The strategy piece includes creating a plan, it might include research (audience research or benchmarking w/peers). For some of you, your pilot might be “prospective” – as part of planning a future campaign or program. Or it might be analyzing the past – something you’ve already implemented but want to go back and measure results/value for planning next reiteration or to do present to senior management to pave the way for more incorporating social media into your program delivery or communications channel. The do part – your tactics, tools – The measure is your data collection – as part of your strategy you define this – it includes analysis – you don’t just collect data, you figure out what it means in terms of improving your results – and then you adapt your plan/strategy and the cycle continues.Make sense? Does this sound familiar? Is this way you measure other programs or communications efforts that do not include social media?
These are the basic steps that you will follow for your measurement pilot …. Or any project that involves social media ..The challenge here is that you are not just measuring, but measuring around your practice or strategy.The pilot is making it very simple.
I’m keeping a public journal too – about the program and using it to reflect on what I’m learning about tracking the program against the SMART objectives. I will be tweaking as we go to get better results …. And welcome you to add your reflectionshttp://measure-netnon.wikispaces.com/Beth-Kanter-Journal
I’m keeping a public journal too – about the program and using it to reflect on what I’m learning about tracking the program against the SMART objectives. I will be tweaking as we go to get better results …. And welcome you to add your reflectionshttp://measure-netnon.wikispaces.com/Beth-Kanter-Journal
http://www.flickr.com/photos/casch/375275440/sizes/o/in/photostream/Fear that it is “not fun” Creating is more appealing then analysis to most people who work in nonprofits. Sadly, many people are more likely to build instead of measure even if they are building on a fault line. That comes as no surprise because most people don’t decide to go into non-profit work because they want to sit and look at spreadsheets all day. Even if there is a staff position for analytics, it tends to be in finance not communications. And not all communications departments invest in professional development in measurement for staff members responsible. 1/in/photostream/
http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/Fear of consequences The nonprofit world can be a scary place in a fragile economy.No one wants to take risks if the results could lose you a promotion or worse, lose your job. So inherently measuring results is a scary prospect. What if we discover that our campaign didn’t get the results we thought it would or even worse, that our precious time was wasted.
http://www.flickr.com/photToo often, there is a fear that by setting objectives, you will be held accountable for some else’s unreasonable expectations. This is the result of “SMART” objectives – which we will discuss in depth in the next chapter. Not setting measurable objectives is akin to running a marathon in an earthquake blindfolded. http://www.bethkanter.org/25-smart/os/alistairhall/5223945711/in/photostream/
http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/There is a common misperception that valid measurement requires highly trained specialists. Or if it is done in house, it is considered time-consuming and just one more chore on a seemingly endless list. However, if nonprofits match their data collection approaches to their capacity, chances are measurement will save time and resources. It is better to start small with a few data points and strategically add metrics to the spreadsheet dashboard. Nonprofits need to think of the measurement task as training for marathon. If you are just starting to get into shape, you need to build up your strength before you run a 35-mile marathon. http://www.flickr.com/photos/josephrobertson/127758523/sizes/l/in/photostream/
http://www.flickr.com/photos/alistairhall/5223945711/in/photostream/Many nonprofits default to the “measure everything” approach on the mistaken belief that more data (or even bogus data (is better than nothing. Worse they collect data “just in case” Either way, these results in bloated spreadsheets to which no one pays attention. so a data-driven organization needs to be comfortable in just saying “No!” to too much data. In a recent interview, Jodi Nelson, who heads up the Bill & Melinda Gates Foundation’s Impact Planning and Improvement Unit, said, “Even when you’ve figured out the metrics, it doesn’t necessarily mean that the data you collect end up getting used. It may sound intuitive, but the key is figuring out how to guarantee the data are both relevant and used. You’d be amazed at how much data collection goes on in the world that produces reports that are never read or used to do anything differently. “http://www.alliancemagazine.org/node/3699
1. Visualize success and failure2. Spend more time identifying what you want to measure, not how to measure it 3. Measure in context – don’t ever collect data unless you have SMART objectives and a benchmark to measure it against.4. Don’t wait until the end of the pilot to gather and analyze your data. Build measurement in from the start and then collect in regular weekly or monthly increments.5. Don’t ever just shovel data over the fence and onto your executive director’s desk. Share high level insights, make recommendations that can spark ideas on improvement6. Less is more. Discipline yourself to measure only one objective or one channel and don’t collect more than data points.7 . Start with a small measurement pilot that helps your organization create a habit of collecting, analyzing, and applying data. We describe in this great detail in the next sections.
Session 3: Why Does Bad Measurement Practice Happen To Good Nonprofits?September 22 at 11:00 am PST/ 2:00 ESThttp://measure-netnon.wikispaces.com/Grp+2+Session+3+Bad+Measurement