1. Peer Learning Group 1:
Measuring the Networked
Nonprofit:
Social Media Measurement
Tool Box
Session 5: April 15, 2013
Beth Kanter,
Visiting Scholar, Social Media and Nonprofits
The David and Lucile Packard Foundation
Organizational Effectiveness Program
2. Welcome!
If you experience any technical
difficulties logging into the
system, please contact Ready/Talk
Customer support: 800.843.9166
Please use *6 to Mute your
conference line
While we are waiting, play with the
chat: Type in your
name, organization, and location. I’ll
do the roll call. Take this time to jot
down notes in your wiki journal.
Only the
moderator
can see you
chats
3. Participants: Roll Call
AAPIP
American Civil Liberties Union
American Leadership Forum - Silicon Valley
Arts Council Silicon Valley
Community Foundation Santa Cruz County
COMPASS
Exhale
GlobalGiving
Grantmakers for Effective Organizations
Ibis Reproductive Health
International Women's Health Coalition
Kuumbwa Jazz
Leadership Learning Community
Leopold Leadership Program
Marine Science Institute
PACT
Population Action International
Roots of Change
Stanford Social Innovation Review
The Encore Fellowships Network (hosted by encore.org)
United Way Silicon Valley
Upwell, incubated by Ocean Conservancy
WildAid
Young Invincibles
*7 unmute
* 6 mute
5. Peer Learning Group 1:
Measuring the Networked
Nonprofit:
Social Media Measurement
Tool Box
Session 5: April 15, 2013
Beth Kanter,
Visiting Scholar, Social Media and Nonprofits
The David and Lucile Packard Foundation
Organizational Effectiveness Program
7. Agenda
Only the
moderator can
see you chats
Tweet your
insights
#netnon
• Roll Call/Recap
• Find the Right Tool for the Job
• Tool Box Share
• Spreadsheet and Dashboard Share: Action
Learning Projects
8. • 50% (12) of participants design, implement, and
document an action learning project that improve results
of social media strategy through measurement
• Participants generate six case studies of how
nonprofits can measure social media effectively that are
published on Beth’s Blog and/or presented during a call
• 50% (12) of participants improve baseline level of
maturity for one or more CWRF Measurement Indicators
and average for group increases by .5 point
Peer Learning Program Outcomes
9. Poll: Defining Your Action Learning Project
Case Study
Project Finished
Implementing, Not
Finished
Designed, Not
Implementing
Started designing
Not Started
12. The Right Tool for the Job
Goal KPI Tools
Increase donations or donors % reduction in cost per dollar
raised
% increase in donors
CRM Database
Increase web traffic and sign ups % increase in referral traffic
% increase in conversion rate
Web Analytics
Increase awareness % increase in awareness,
% increase in
visibility/prominence
Survey
Improve relationships with
existing donors/volunteers
% improvement in relationship
scores,
% increase in donation from
existing donors
Survey
Improve engagement with
stakeholders
% increase in engagement
(comments on YouTube, shares on
Facebook, comments on blog, etc.
Web and Social Media Analytics
Change in behavior % decrease in bad behavior,
% increase in good behavior
Survey
Communicate Messages % of articles containing key
messages
% of mentions on message or
positive/negative
Percentage aware of or believing
key message
Media Content Analysis
Survey
13. • Sentiment
• Themes
• Messaging
Content
Analysis
• Attitudes
• Preferences
• Behavior
Survey
Research
• Reach
• Engagement
• Action
Analytics
Three Categories of Tools
14. Media Content Analysis Tools
Authority
Visibility
Tone
Messages
Communicated
Conversation
Type
Chapter 8
Goal: Communicate Messages % of articles containing key
messages
% of mentions on message or
positive/negative
19. Manual Content Analysis
Pages 133-137
Analytics ToolsSocial Network Analysis Tool: NodelXL
Network Metrics Project
Goal: measure our network in at
least two new ways using NodeXL
by June 1
Objectives:
Test a way to assess the health and
strength of network relationships
over time
Test hypothesis that our work
strengthened network relationships
Train at least two team members in
social network analysis (SNA)
Share results publicly
28. Spreadsheet and Dashboard Share
• Roots of Change
• Global Giving
• Community Foundation of Santa Cruz
• Compass
• Leopold Leadership
• Leadership Learning Community
30. CA FOOD POLICY COUNCIL
- List of Key Influencers -
Our list includes:
•Councils
•Organizations
•Individuals/ bloggers
31. Who’s responsible?
What’s the communication channel?
By When?
Who are you communicating to?
CA FOOD POLICY COUNCIL
ADVOCACY DAY Good Food For All
Learn – Engage -Advocate
32. Action Learning Project: Peer Tool Share
Organization Name: GlobalGiving
Action Learning Project: Pro-active Twitter Engagement Experiment
Goals KPI Tools
To determine whether pro-active
engagement with donors on twitter elicits
higher quality conversations on twitter
(and higher engagement rates)
% increase in conversation
quality
Homemade tools
(spreadsheet) to track
conversation quality
[creation in progress]
To determine whether pro-active
engagement on twitter (and higher quality
conversations + higher engagement rates)
turns donors into better advocates for
GlobalGiving on twitter
% increase in engagement
rates (aggregate of:
applause rate, conversation
rates, & amplification rates)
Argyle Social
To increase donations from this target
group of donors.
% increase in donations
over control group
Argyle social (publishing
with google url parameters)
and google analytics and
our own donation database
• What is your project? An experiment in pro-actively engaging with our donors on twitter
over a period of 4-6 months
• Where are you in implementation or design? Still designing tools and forming test groups.
33. Action Learning Project: Pick One Tool to Share
• Name of the tool/ Free or Paid tool?
• Argyle Social
• What do you like about this tool?
• It tracks most of the metrics I want for twitter
• What data points does it collect that help you
understand whether your reached your goal or match
your KPI?
• Applause rate (clicks on urls in posts)
• Amplification rate (RTs)
• Conversation rate (replies)
34. Inside Argyle Social’s dashboard
In theory, you can see all the info you want on a per-post level:
• Applause rate (clicks)
• Amplification rate (RTs)
• Conversation rate (replies)
• Donation rate ($ - questionable accuracy.)
35. Argyle Social’s downloadable reports (not as helpful!)
For some reason the data dump is not as helpful. It’s not at a per-post level anymore,
and there are no click rates available in data download.
I still have to do a lot with this manually to get it into the form I want.
37. Action Learning Project: Peer Tool Share
Organization Name: Community Foundation Santa Cruz County
Action Learning Project: Philanthropy 831 Blog
Where are we? We have a communications (and blog) strategy and a content
curation calendar. We’re blogging weekly and are now working to refine tracking
methods and a simple dashboard for reporting.
38. Action Learning Project: Peer Tool Share
Organization Name: Community Foundation Santa Cruz County
Action Learning Project: Philanthropy 831 Blog
Goals (from Jan 1 to June 15) KPI Tools
Increase subscribers Minimum 50% increase in RSS
subscribers, from 1-4 to 2-6,
per blog category.
Feedburner
Increase traffic 50% increase in Unique
Pageviews, from 461 to 690.
Google Analytics
Increase engagement Minimum 50% increase, from
0-2 to 1-3 comments, ratings or
shares per post.
Feedburner, Google
Analytics, Hootsuite
& Website
39. Action Learning Project: Philanthropy 831 Blog
Primary Tool(s): Free Analytics (Google, Feedburner, HootSuite
and website’s blog page)
Why? Easy to use, rich data and free. We think it makes sense
(and not too much) to look at all four sources above, not just
one.
Data Points we’ll track: Subscriptions, Traffic flow (Where did
they come from?), Visitors and Page Views and
Comments/Ratings (What did they do when they got here?)
and Clicks (Where did they go?).
How we’ll organize: Create customized dashboard that
aggregates this data and measures engagement.
40. Action Learning Project: Philanthropy 831 Blog
Our plan is to customize the data collection from Google Analytics on a
page like this and aggregate it with the other analytics in a simple
dashboard coming soon! We promise.
41. Manual Content Analysis
Pages 133-137
Action Learning Project
Goals KPIs
Visibility % increase average weekly unique page views
% increase in Twitter followers
Value % increase in subscriptions
% increase in RTs on Twitter
Engagement % increase in comments on blog
% increase in @ mentions on Twitter
43. Action Learning Project: Leopold Leadership Program
Outcomes KPI
Create Number of blog posts per month
What is a doable publishing schedule?
What is the frequency of publication?
Word length
Past data from newsletters – teaching/learning
How many writers: ~dozen
Consume Unique views of blog posts
Subscribers
Amplify Tweets
Retweets
Engage Comments on posts
Apply content User surveys
Share lessons Number of blog posts or comments that discuss how
the writer applied learning
Ladder of Engagement
44. Tools for Action Learning Project: Leopold Leadership
Program
Templates help us:
• manage work flow
• design for measurement
• know which formats work
& why with our readers
45. Action Learning Project: Leadership Learning Community
We are a national nonprofit
organization transforming the way
leadership development work is
conceived, conducted and
evaluated, primarily within the
nonprofit sector. We focus on
leveraging leadership as a means to
create a more just and equitable
society.
We are going to focus our project on
our upcoming national
meeting, Creating Space.
46. Goal 1: Registration
Primary channel is
email. Testing calls to
action from emails -
what works best to
convert to subscription
Register 80 People
Goal 2: Build Email List
Promote subscription
through social media,
blog, webinars
Grow List to 4,000
(currently at 3,500)
12% increase
Goal 3: Influencer
Give registration codes
to design group and
encourage them to
promote to their
channels, track
conversions
Drive 20 Registrations
Goal 4: Engagement
Twitter chat before the
event, promote the
hashtag during and
after the event
50% of registered
attendees engage
before, during, and
after
Description KPI
48. Action Learning Project: Reflection
• What do you need to implement your action
learning project?
Only the
moderator can
see you chats
Type into Chat
*6 mute
* 7 unmute
49. May 29, 2013
1:00 PM PST
Wednesday
Continue to implement action learning project
Topic: Making Sense of Your Data
Need help? Book a coaching session (I’m adding dates)
https://my.timedriver.com/4QHZG
Next Session
Editor's Notes
Welcome. This is the very session for this project and I’m thrilled that you have decided to participate in this learning journey. I look forward to learning a lot from you. Today’s call is an orientation to the program and an opportunity for you to ask questions.Give my gratitude the The David and Lucile Packard Foundation for supporting this project and my work …AgendaRoll Call/RecapFramework for Selecting Measurement ToolsTool Box ShareSmart Spreadsheet TricksROC - (Tracking Spreadsheet for influencers and content)Leadership Learning Community (Tracking/Spreadsheet)Upwell - Nodel XLMarine Science Institute - SurveyNext StepsHow To Be Prepared:Keep moving on implementation of your action learning project - book a coaching session with Beth Kanter if you need some helpHomework: Fill in the 2-3 PPT slides in this template (it won't take you long!)What measurement tools are you using and why?What data does it collect to help you understand whether you have reached your goal?Share your spreadsheet (not the data, but the structure)
Every few minutes as we get started, tech support reminder, type into the chat, roll call
Because this is our first call, I’m going to run down the names of the organizations and have everyone say their name – so we can hear each other’s voices. I won’t be doing this for every call – we will get into grove where you should arrive 5-10 minutes early and announcement yourself. If you arrive after the call is underway, let us know you are hear by using the chat.
http://www.flickr.com/photos/malinki/2621920871/sizes/o/Start recording about 2 minutes late to let people join *2
Welcome. This is the very session for this project and I’m thrilled that you have decided to participate in this learning journey. I look forward to learning a lot from you. Today’s call is an orientation to the program and an opportunity for you to ask questions.Give my gratitude the The David and Lucile Packard Foundation for supporting this project and my work …AgendaRoll Call/RecapFramework for Selecting Measurement ToolsTool Box ShareSmart Spreadsheet TricksROC - (Tracking Spreadsheet for influencers and content)Leadership Learning Community (Tracking/Spreadsheet)Upwell - Nodel XLMarine Science Institute - SurveyNext StepsHow To Be Prepared:Keep moving on implementation of your action learning project - book a coaching session with Beth Kanter if you need some helpHomework: Fill in the 2-3 PPT slides in this template (it won't take you long!)What measurement tools are you using and why?What data does it collect to help you understand whether you have reached your goal?Share your spreadsheet (not the data, but the structure)
Here’s a little bit about me – blogger, author, trainer.A lot of my work lately has been designing and facilitating peer learning networks about becoming networked nonprofits and social media– the photo there is a cluster of Packard Fdn. Grantees that focus on family planning … I was in Delhi in June for the start up – an intensive boot camp, followed by remote assistance. There’s were great lunches there, so to avoid people falling asleep … I made them move. The hotel had beautiful three story staircase and they had do laps … so if you do training – incorporating movement and interaction helps people learn and we’re going to do a lot of that today!
AgendaRoll Call/RecapFramework for Selecting Measurement ToolsTool Box ShareSmart Spreadsheet TricksROC - (Tracking Spreadsheet for influencers and content)Leadership Learning Community (Tracking/Spreadsheet)Upwell - Nodel XLMarine Science Institute - SurveyNext StepsHow To Be Prepared:Keep moving on implementation of your action learning project - book a coaching session with Beth Kanter if you need some helpHomework: Fill in the 2-3 PPT slides in this template (it won't take you long!)What measurement tools are you using and why?What data does it collect to help you understand whether you have reached your goal?Share your spreadsheet (not the data, but the structure)
Categorize your specific social media measurement activities and relate to your objectivesSentiment (Messaging, positioning, themes)Attitudes (perceptions, behavior change, preferences, awareness)Do (Reach, Engagement, Action, Donate, Purchase)http://www.flickr.com/photos/leeontheroad/89666692/sizes/z/in/photostream/
Network Metrics ProjectGoal: measure our network in at least two new ways using NodeXL by June 1Objectives:Test a way to assess the health and strength of network relationships over timeTest hypothesis that our work strengthened network relationshipsTrain at least two team members in social network analysis (SNA)Share results publiclyActivities to date:Used NodeXL to map an overfishing conversation on twitter, and included this analysis in final report to the Waitt Foundation.Two team members individually attended NodeXL open office hours on G+.Introduced Marc Smith of the Social Media Research Foundation to Allen Gunn and Aspiration Technology. Facilitated an invite for Marc to the Strategic Technology Teach-in to be held on T, 3/26. Signed up to assist Marc at his SNA workshop for the Predictive Analytics conference.Worked with Marc to map the #netnon twitter hashtag, the full results of which are hosted here - a static image of which is pasted below.http://nodexl.codeplex.com/
ROC has created a list of key influencers to identify who in the CAFPC network would be a good ambassador for the CAFPC Advocacy DayExcel identifies whether or not they have a presence on FB and Twitter, if so how large is their followingThen we take it one step further by identifying select members within each of the councils and we track whether or not they have a presence on FB and Twitter, if so how large is their following
Homework: Create a PPT slide with the following:What measurement tool(s) will you use to collect and organize your data? Why?Include a screen capture of the tool -- and you plan for cleaning, analyzing, and organizing the data to get insightsShare your spreadsheet (not the data, but the structure)
Homework: Create a PPT slide with the following:What measurement tool(s) will you use to collect and organize your data? Why?Include a screen capture of the tool -- and you plan for cleaning, analyzing, and organizing the data to get insightsShare your spreadsheet (not the data, but the structure)
Homework: Create a PPT slide with the following:What measurement tool(s) will you use to collect and organize your data? Why?Include a screen capture of the tool -- and you plan for cleaning, analyzing, and organizing the data to get insightsShare your spreadsheet (not the data, but the structure)
Homework: Create a PPT slide with the following: What measurement tool(s) will you use to collect and organize your data? Why?Include a screen capture of the tool -- and you plan for cleaning, analyzing, and organizing the data to get insightsShare your spreadsheet (not the data, but the structure)
Homework: Create a PPT slide with the following: What measurement tool(s) will you use to collect and organize your data? Why?Include a screen capture of the tool -- and you plan for cleaning, analyzing, and organizing the data to get insightsShare your spreadsheet (not the data, but the structure)
SMART OBJECTIVE 1: Increase blog visibilitySMART OBJECTIVE 2: Increase value of our content to readersSMART OBJECTIVE 3: Increase community engagement with our postsWe want our blog to become the core of a engaged community of scientists who are committed to science outreach and engagement with the media and policy worlds. Specifically, we want more traffic, more subscribers, an increased number of commenters, and other evidence of a strengthened connection between these researchers and our team, and among the researchers. We hope they come for the content and stay for the community. This project directly supports our mission of helping scientists build the skills & relationships they need to effectively share their knowledge in broader social conversations.We are most keenly interested in engaging academic scientists in the environmental sciences. Grad students bring deep enthusiasm and energy, early career researchers can speak from recent personal experience on the tenure-track career, and mid-career and senior researchers have the authority and resources to make change within their institutions. Their social media use varies greatly, but part of our role in developing our network is to support greater familiarity with online conversations. For a sizeable number of scientists, we were their first real introduction to social media. For a small number of them, we continue to be a main conduit to that worldList Your Metrics:• Visibility:o % increase in average weekly unique pageviewso % increase in Twitter followers• Value:o % increase in subscriptionso % increase in RTs on Twitter• Engagement:o % increase in comments on blogo % increase in @ mentions on Twitter
SMART OBJECTIVE 1: Increase blog visibilitySMART OBJECTIVE 2: Increase value of our content to readersSMART OBJECTIVE 3: Increase community engagement with our posts
This is our full ladder of engagement.For this project, we’ll be focusing on the first rung – the outcomes & KPI shown in blue.Once we’ve gone through a full month or so of publishing and measuring, we’ll use the data analysis (from “create” + some “consume”) to refine our editorial calendar & process As soon as we’re ready, we’ll move to the “Amplify” phase and incorporate Twitter & email subscription drives into the content creation processAll along, we’ll be looking for evidence of engagement, application of our content, and sharing as noted above.