Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

NPC, Measuring Impact in the Voluntary Sector 2016


  • Login to see the comments

  • Be the first to like this

NPC, Measuring Impact in the Voluntary Sector 2016

  1. 1. v MEASURING SOCIAL IMPACT Cecilie Hestbaek, NPC The Big Connect, Tuesday 7th June 2015
  2. 2. NPC: TRANSFORMING THE CHARITY SECTOR 2 NPC works at the nexus between charities and funders Charity SectorFunder Increasing the impact of charities eg, impact-focused theories of change Strengthening the partnership Eg, collaboration towards shared goals Increasing the impact of funders eg, effective commissioning Consultancy & Think tank
  3. 3. SOCIAL IMPACT ‘The effect of an activity on the social fabric of the community and well-being of the individuals and families.’ 3
  4. 4. a) articulate how you think a service or intervention will have an impact; b) and test your thinking against the evidence you have available. 4 THE ESSENCE OF IMPACT MEASUREMENT: Exercise: Take 10 minutes to discuss with your neighbour why it’s important to measure social impact and what the challenges are. Write key words on different colour post-its.
  5. 5. WHY IS MEASURING IMPACT IMPORTANT? Saves staff time Influences the debate on “what works” Improved services Raises profile Motivates staff / volunteers Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater Impact. New Philanthropy Capital. Helps secure funding 5
  6. 6. CHALLENGES IN MEASUREMENT 6 Causality, consistency and predictability. • What worked before might not work again because the external environment has changed Time frame • Change can happen quickly; or it might take two decades to come about Contribution and attribution • A variety of external factors are beyond your organisation’s control and you will often have no counterfactuals Data collection • If you are targeting top-level decision makers, it can be difficult to get an answer from them about why they did (or did not) change their mind on a particular issue. …and ensuring that you measure outcomes, not outputs.
  7. 7. THE FOUR PILLARS APPROACH TO MEASURING IMPACT 7 Map your theory of change Prioritise what you measure Choose your level of evidence Select your sources and tools Effective measurement framework developed Strategic vision Leadership Case for impact measurement
  9. 9. WHAT IS A THEORY OF CHANGE? Links activities intermediate outcomes  final outcomes A description of how activities lead to outcomes 9 - Clarifies what the activities aim to achieve and how - Provides the case for why achieving intermediate outcomes is important - Provides a structure for identifying what can be measured
  10. 10. HOW TO REPRESENT A THEORY OF CHANGE 10 Planning Triangle Logic Model Outcomes Chain However you represent your theory of change, it should be supported by a written description.
  11. 11. 11 Children’s emotional resourcefulness improves Counselling Clients’ ability to support their children's healthy development improves Clients’ emotional or psychological difficulties decrease Mother / Child interaction improves Clients’ capacity for self care increases EXAMPLE (SIMPLIFIED) THEORY OF CHANGE MOTHERS’ COUNSELLING SERVICE Activities Intermediate Outcomes Final outcome
  12. 12. EXERCISE 12 • Take 2 minutes to think about 1-2 social outcomes that are central to what you do. • Write them down on a piece of paper. • As we go through the rest of the presentation, try to reflect on the measurement of that/those outcomes. How would you apply the theory and measure those? Categories of outcomes include: ‘soft’ outcomes, such as attitudes, knowledge, skills, behaviour; and ‘hard’ outcomes, such as employment, educational attainment, reduced reoffending etc.
  14. 14. PRIORITISE OUTCOMES THAT: • are directly influenced (rather than indirectly supported) • are important / material to the mission • are not too costly to measure • will produce credible data 14
  16. 16. COUNTERFACTUAL • Comparing the world with your organisation in it with what the world would be like without it. • Control group • Attribution 16
  17. 17. LEVELS OF EVIDENCE 17 Randomised control trial Anecdotes / quotes Before and after survey Self-reported change Case studies Control groups Credibility Basic Advanced
  19. 19. • Quantitative data (numbers) • Statistical estimates • Prevalence of views, attitudes and experiences • Admin data or questionnaires (paper, web, etc.) • Qualitative data (words) • Detailed understanding • In-depth interviewing (telephone or face-to-face) • Observation • Stakeholders’ views Proportion of beneficiaries whose outcomes have improved, and by how much. What did beneficiaries think, did it make a difference to them? How? How could it have been better? DIFFERENT TYPES OF EVIDENCE
  20. 20. EVALUATION TRAPS 20 Collect data that matters, and work together Don’t force squares into circles & don’t collect arbitrary data
  21. 21. INSPIRING IMPACT: MEASURING UP! AND THE IMPACT HUB Measuring Up! • online, step-by-step self-assessment tool • looks at the way you plan, evidence, communicate and learn from the difference your work makes The Inspiring Impact Hub • pulls together resources relevant to improving impact practice • enables users to search and filter results according to their needs 21