Measuring Social Change & Media: Beyond BSSXSW 2013: Beth Kanter, KD Paine, David J. Neff, Carie Lewis
Beth Kanter: Master Trainer, Author, and Blogger @kanter
Katie Delahaye Paine Publisher The Measurement Standard News Group International firstname.lastname@example.org www.themeasurementstandard.com http:/kdpaine.blogs.comFellow and Board Member: Society for New Communications Research IPR Measurement Commission 3/7/2013 www.themeaasurementstandard.com
David J. Neff, Author and creatorLights. Camera. Help. / SXSW2013@daveiam@npfilm
Who are you?Raise your hand if …….- Social Media Manager for Nonprofit- Social Media Manager for For-Profit- Other Nonprofit Staff- Consultant/Social Media- Consultant/Measurement or Evaluation- Other
And your Org?Raise your hand iforganization is budget is ..-Less than $1 million-$1-$5 million-Over $5 millionType ..-Social Service-Environment-Arts-Education-Animal Welfare-Community Services-Health Care-Other
Workshop Agenda AGENDA OUTCOMESIntroduction Understand how to get started with a measurementData Informed program for yourOrganization organization’s social mediaCase Studies FRAMING7 Steps ofMeasurement InteractiveCase Studies FunWrap Up #NETNON
Raise Your Hand If Your Strategy Goal Is …. Improve relationships Change behavior Increase awareness Increase engagement Increase dollars Increase action
Why did it fail?What did we learn?What insights canuse next timearound?DoSomething.Org’s Fail Fest
CWRF: Becoming Data Informed: What Does It look like? Crawl Walk Run Fly Lacks consistent data Data collection Data from multiple Org Wide KPIs collection consistent but not sources shared No reporting or Data not linked to System and structure Organizational synthesis results, could be for data collection Dashboard with wrong data different views, sharing Decisions based on gut Rarely makes decisions Discussed at staff Data visualization, to improve meetings, decisions reporting, formal made using it reflection processAnalysisToolsSense-Making
[HSUS Is Data Informed]HSUS is the nation’s largest and most effective animal welfare organization. Most Important KPI’s: • Amount raised • # of action takers • # names new to file • Growth rate • Customer service wins
[Not Every Department Is Data-Informed] 2011: FEAR stage Web: 1% of donations2013: DATA INFORMED stage Web: 33% of donations
THE DATAINFORMED ARTSORGANIZATIONBy David J. Neff, Author andcreatorLights. Camera. Help. / SXSW2013@daveiam@npfilm
Measurement Indicators: Analysis CRAWL WALK RUN FLY Lacks consistent data Data collection is Data is from multiple Establishes organizational collection or formal consistent, but not sources and shared KPIs and tracks in reporting. Draws shared between across departments organizational dashboard conclusions from departments. Not all through a dashboard. with different views for incomplete data or data is linked to Does not collect data it departments or levels. “drive by” analysis. decision-making for doesn’t use. May have data analyst on better results. Measurable objectives staff. are based on benchmarking. LCH HSUS
[ 2 = WALK Stage for Analysis] (Some people are still not all about it and resist the change in process )Our Analysis Process: Our Challenges:• Everyone gathers their data • Reports aren’t distributed org-wide yet• We sit in a room and analyze it • Still making some decisions based on what• The project manager develops a report of we think, not supported by data “what we did/what we learned/what we • Upper management still not on board, just recommend for next time “+ data wants things to get done
Maturity of Practice: CWRF - Tools CRAWL WALK RUN FLY Not using or not Using free or low Using free/low cost Uses professional using fully. cost analytics tools analytics tools to measurement and to collect metrics collect metrics and analytics tools. and analyze further analyze further in Provides training or in spreadsheets if spreadsheets if uses expert required for required for consultants to assist actionable insights. actionable insights. in data/analysis. Uses social media management/metric s professional tool to collect data. HSUS LCH
[ 3- RUN Stage for Tools ]• We have the tools in place• We’ve trained people on how to pull the data• We’re using several different tools• We’re keeping a running spreadsheet of data
Maturity of Practice: CWRF – Adjustment CRAWL WALK RUN FLY Does not use Uses data for Reports are Formal process data to make decision-making discussed at for analyzing, planning but not a formal staff meetings discussing, and decisions. organizational and used to applying results. process. make decisions Data that improve visualization and results. formal reflection processes. LCH HSUS
[ 3 – RUN Stage for Adjustment ] How we’ve adjusted:• Reports are discussed• Reports are revisited before making campaign plans the next year• Decisions are based on previous reportsExample: Hurricane Sandy• Sharing fundraising worked really well, will continue to optimize this technology• Text to give featured as prominent giving vehicle in social media for the first time• Did very well working in conjunction with donation link on Facebook photo album• Will use again – in disaster planning will set up social media text to give code immediately
ADJUSTMENT :WE REPEAT THE MISTAKES OF THEPAST + = 2
[ Some Advice… ]1. Patience. It was so hard to get everyone on board that “data matters.” Be patient.2. Baby steps. We tried to do too much and ended up right back where we started.3. Put together a digestible, ONE PAGE report on your findings.4. Figure out the technical stuff. It may take forever, but you’ll be glad you did.5. Campaign. Teach others why data and analysis is so important.6. Develop a tracking system. Create a naming convention for your tracking links, and hold people accountable7. Narrow it down. Again, don’t try to do too much. Figure out what matters.8. Share your successes! Give stakeholders the report, don’t hoard it!
TOP FIVE PIECES OF ADVICE 1) Make it a priority – It’s not just your team in the process 2) Organize – Call an all staff meeting or get it part of a future all staff meeting 3) Plan using a tool – Find a framework you want to use. Such as the OGST format 4) Involve your volunteers/board – What data driven professional can help you solve this issue? Think mgmt consultants, Programmers, Data designers. 5) Experiment with the tech – Don’t end with Excel. Fail forward with the best tool for your level of data collection. Pivot Tables 101 anyone?
Share Pair: Where is Your Organization? Crawl, Walk, Run or Fly?
It’s not about you, so get over it ConversationsMember Development Research Volunteers Funders HR RecipientsService Savings, shorter cycles, more renewals, better ideas, research 3/7/2013 www.themeaasurementstandard.com
The 7 steps to Social Media MeasurementStep 1: Define your goal(s). What outcomes is this strategyor tactic going to achieve? What are your measurableobjectives?Step 2: Define your audiences. Who are you are trying toreach? How do your efforts connect with those audiencesto achieve the goal.Step 3: Define your investments. What is it really costingyou to achieve this outcome?Step 4: Define your benchmarks. Who or what are yougoing to compare your results to?Step 5: Define your metrics. What are the indicators tojudge your progress?Step 6: Select your data collection tool(s).Step 7: Analyze your data, turn it into action, measureagain 3/7/2013 www.themeaasurementstandard.com
Step 1: Define the goals: Why Social Media? What return is expected? – Define in terms of the mission. What problems is Social Media supposed to solve? What were you hired to do? What difference are you expected to make? If you are celebrating complete 100% success a year from now, what is different about the organization? If your Social Media is eliminated, what would be different? 513/7/2013 www.themeaasurementstandard.com
Goals, Actions and Metrics Goal Action Activity Metric Outcome MetricIncreased Conduct a Number of likes & % increase in donationsDonations photo contest share of contest Cost per new donor link acquiredIncrease Modify % increase in traffic % increase inmembership content, add to web site membership as share of interactivity income functions3/7/2013 www.themeaasurementstandard.com
Step 2: Don’t ask me, ask your stakeholders What you need to listen for: – What keeps them up at night? – What are they currently seeing? – Where do they go for information? – What influences their decisions? – What’s important to them? – What makes them act? 533/7/2013 www.themeaasurementstandard.com
Step 3: Establish benchmarks Past Performance Think 3 – Peer – Underdog competing for share of wallet – Stretch goal Whatever keeps the Board suite up at night3/7/2013 www.themeaasurementstandard.com
Step 4: Why you need a Kick-Butt Index The Perfect KPI Is actionable Is there when you need it Specific to your priority Continuously improves your processes Gets you where you want to go You become what you measure, so pick your KPI carefully 55 3/7/2013 www.themeaasurementstandard.com
Step 5: Define your investment Be honest – Social Media is not “free” Be transparent Don’t forget opportunity costs The CFAC example3/7/2013 www.themeaasurementstandard.com
Step 6: Pick the right measurement tools If you want to measure messaging, positioning, themes, sentiment: Content analysis If you want to measure awareness, perception, relationships, preference: Survey research If you want to measure engagement, action, purchase: Web analytics If you want predictions and correlations you need two out of three 573/7/2013 www.themeaasurementstandard.com
Step 6: Selecting a measurement toolObjective KPI ToolIncrease inquiries, web traffic, % increase in traffic Web Analytics: Google Analytics,recruitment #s of clickthrus or downloads Omniture, Web trendsIncrease awareness/preference % of audience preferring your brand Survey: Phone Calls, SurveyMonkey, to the competition or MailEngage marketplace Conversation index greater than .8 Web analytics or Content Analysis: Rankings Facebook Insights, Convio, Omniture, % increase in engagement Google AnalyticsCommunicate messages % of articles containing key Media content analysis, Survey messages Research Total opportunities to see key messages Cost per opportunity to see key messages 58 3/7/2013 www.themeaasurementstandard.com
The ROI of Emily 593/7/2013 www.themeaasurementstandard.com
So What = Revenue 35,152,789 OTS Red line indicates media 6,253,852 OTS impressions 603/7/2013 www.themeaasurementstandard.com
Thank You!For more information on measurement, read my blog:http://kdpaine.blogs.com or subscribe to The Measurement Standard:www.themeasurementstandard.comFor a copy of this presentation go to: http://www.kdpaine.comFollow me on Twitter: KDPaineFriend me on Facebook: Katie Paine3/7/2013 www.themeaasurementstandard.com
[ 7 Steps of Measurement – Hurricane Sandy ] Goal: Audience: Investment: Raise $x million to people who love staff time, support animal animals and want technology hosting relief efforts to help Metrics: Tools: Benchmarks: amt. of donations, Convio, Tweetdeck, last hurricane donations by Facebook Insights, (Isaac) we raised $x source, Twitter ad platform thousand sentiment % Insights: text to give fundraising in social media works really well so we need to do more of it, we raised a significant amount of money via sharing so we need to continue to optimize that medium, Facebook photo album tied to donations worked because people are so visual so we need to use photos more to tell our story.
BENCHMARK Adoption is hard to track (multiple shelters/locations/city vs nonprofit) Varied programs that have varied results across the United States. No previous adoption program that contained all the metrics and goals that we would be experimenting with.
METRICS Once again, adoption is hard to track What can we track to get people to think about adoption?- Pledges to Adopt- Pledges to visit a shelter- Pledges to walk a dog at a shelter- Pledges to tell their friends about adoption on social channels- Pledges to take a photo of a dog at a shelter- Join a fundraising event in your area
TOOLS HTML/CSS for Site iPhone = C++ Android = Java Vwalls = Java Social Channels = FB, Twitter, YouTube, Later Foursquare Meetup.com. Wait? People still use that? YES. Measuring with TweetReach and SproutSocial
INSIGHT 1) Should of defined a tighter audience with personas. 2) Undertaking dual app development in concurrent work streams is a GIANT PAIN IN THE ASS. 3) Working with Apple to get an app in the app store is a GIANT PAIN. 4) People loved sharing user generated content. We even had people create meme’s around invisible dogs. Community for BFAS is inspiring. 5) App market is crowded. Needed to spend more budget towardsmarketing the adoption app.
It is your turn to apply the 7 steps!Step 1: Pick A Result Improve relationships Change behavior Increase awareness Increase engagement Increase dollars Increase action