Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CKX: Social Impact Measurement Around the Globe

478 views

Published on

How do we know the effect of our policies, programs and investments? By measuring it. We’re not always very good at it, we haven’t been doing it for long enough, or in enough fields, or with enough collaborations, but we’re getting better and more excited and there’s a whole lot going on in this space right now. In this presentation, Emma Tomkinson, a social impact analyst from Sydney, Australia, will showcase examples of collaborative work in social impact measurement from around the globe.

Published in: Government & Nonprofit
  • Be the first to comment

  • Be the first to like this

CKX: Social Impact Measurement Around the Globe

  1. 1. Demonstrating change is caused by an intervention SOCIAL IMPACT MEASUREMENT Telling a story of what you do and why it matters Count or proportion of social change occurring Using data to make decisions Performance management: continuous evaluation and improvement
  2. 2. (Tris Lumley, NPC, April 2014) FIELD Impact evidence ORGS Managerial / administrative data PEOPLE Constituent Voice What are your priorities / needs / desires? Survey Panel What do we know? Field-level evidence / ‘what works’ What should we do? Programme design How are we doing compared to others? Shared How are we doing? What should we change? Performance management How can we serve you better? Measurement / Benchmarks Data Labs How should we collaborate? Field-level collaboration Pooled funding Recommended practices What should our priorities be? How are we doing collectively? What do we need to change? What have we learned that will improve services? What works for whom, in what circumstances? Impact Evaluation How can we serve you better next time? Constituent voice / Feedback How did we do? Where did we fail? How do we share what we’ve learned? Summative evaluation BEFORE DURING AFTER Field-level Priorities Constituent voice / Feedback
  3. 3. 1
  4. 4. 2 22 2 2 2 2
  5. 5. siaassociation.org
  6. 6. Consultancy; 15% Public sector; 14% Charity; 17% Nonprofit, NGO or community organisation but not charity; 19% Social enterprise; 12% Private sector; 10% Academic institution; 10% International mining company ; 0% Network, association or membership organisation; 3%
  7. 7. 5 4 3 2 1 0 Social enterprise Private sector Nonprofit, NGO or community organisation but not charity Public Consultancy Charity Academic institution Network, association or membership organisation Sliding scale where 5 is most expert I do it for my program or organisation I do it for other programs or organisations Responses to the question: I can identify measures of social impact relating to a program or organisation
  8. 8. 69 62 29 25 18 16 15 10 5 n=109
  9. 9. 2 2 2 2 2 3 3 3 3 3 3 4 5 6 6 7 7 7 7 8 14 47 SROI Surveys Interviews Outcomes Star Theory of Change Bespoke methods/tools Data analysis Logic Model Before and after comparison Results Based Accountability Cost-Benefit Analysis London Benchmarking Group2 Outcomes-based evaluation Warwick Edinburgh Scale HACT Social Value Tool Focus Groups Validated scales Administrative data Outcome Mapping External evaluation de Bono's Thinking tools Results Framework
  10. 10. #opendata 3
  11. 11. (John Burrett, haikuanalytics , 2014)
  12. 12. (John Burrett, haikuanalytics , 2014)
  13. 13. (John Burrett, haikuanalytics , 2014)
  14. 14. 4
  15. 15. -71% -84% -42% -45% -48% -12% -15% -24% -25% -26% 13% 5% 32% 24% 38% 57% 57% 53% 77% 72% Hospitals and GPs relating to care Tax and benefit records to catch fraud Pharmaceutical companies share with academic researchers Email and internet search traffic monitoring to identify terrorists GP health records shared with researchers Energy companies to predict energy needs Health records shared with private healthcare companies Technology companies monitoring searches for flu epidemic Online retailers to target advertisements Health records sold to privae healthcare companies Should not happen Should happen (Royal Statistical Society UK, 2014)
  16. 16. Delivering services to people that need them without a performance management system is like driving a car with no instruments on the dashboard. With the windscreen obscured by ice. You know where you want to go, but you have no idea if you’re getting there.
  17. 17. Service delivery Data Daily data entry onto a shared IT system Analysis Analyst support to collate and analyse data in order to produce data dashboards. Review meetings Regular meetings to discuss dashboards, resource allocation and service improvement. 1. Consent for data collection and use 2. Constructing a shared case management IT system 3. Building capacity of all service delivery workers to collect and use data 5. Feedback loops and reporting 6. Adjusting services in response 4. Performance analysts (adapted from Social Finance)
  18. 18. (http://dcpni.org/)
  19. 19. 5
  20. 20. Pay for performance Comparing intervention participants to a control group ATTRIBUTION COHORT Pay for success Payment by Results Performance-based contracting Outcomes-based contracting • UK Department of Work and Pensions Innovation Fund: disadvantaged young people • UK It’s All About Me: adoption • Manchester: children in care • Granite School District: early childhood education • Saskatchewan – single mothers • Peterborough: ex-offenders • Massachusetts: high risk young men • New York State: employment for ex-offenders • New South Wales Benevolent Society Social Benefit Bond: families with children in care Count or proportion of social change among participants CONTRIBUTION INDIVIDUAL language used
  21. 21. (Ministry of Justice UK, Justice Data Lab)
  22. 22. No 63 68% Yes 30 32% Statistically Significant? increased 5 17% reduced 25 83% Effect on reoffending? (Ministry of Justice UK, Justice Data Lab Oct 14)
  23. 23. 4,500 4,000 3,500 3,000 2,500 2,000 1,500 1,000 500 0 Treatment size -40 -30 -20 -10 0 10 20 Effect on one year proven re-offending rate (ppts) [all statistically significant requests published to date] (Ministry of Justice UK, Justice Data Lab Oct 14)
  24. 24. 0.01 0.1 1 10 100 reduces crime increases crime (Petrosino (2013) Scared Straight Update)
  25. 25. 6
  26. 26. health cochrane.org other areas campbellcollaboration.org
  27. 27. 7
  28. 28. 8
  29. 29. Partners Funders
  30. 30. 9
  31. 31. 10
  32. 32. outcomesstar.org.uk/
  33. 33. emmatomkinson.com @emma_tomkinson

×