Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Anne Kazimirski 2013 Future Focus Workshop: Benifits of Monitoring & Evaluation


Published on

Anne is Deputy Head of the Measurement and Evaluation at New Philanthropy Capital (NCP) and helps charities and funders to measure and communicate their impact. Her role includes developing tools and approaches for improving impact measurement for a wide range of organisations.

VAL was delighted to welcome Anne to present a workshop during our 2013 Future Focus Conference. Anne's workshop was all about helping charities tell a compelling story about what they do and the impact they have.

Specifically, Anne's workshop looked at the benefits of measuring impact, information about the 'theory of change process' to help charities understand what outcomes they are aiming to achieve, and helped groups start thinking about the type of data they need and how best to collect that data.

While the 2013 Future Focus conference is now over, VAL runs trainings and workshops year-round. If you'd like to learn more about training for your organisation, visit

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Anne Kazimirski 2013 Future Focus Workshop: Benifits of Monitoring & Evaluation

  1. 1. BENEFITS OF MONITORINGvAND EVALUATION Anne Kazimirski VAL future focus VCS conference, 18th September 2013
  2. 2. NPC: TRANSFORMING THE CHARITY SECTOR Charity Increasing the impact of charities eg, impact-focused theories of change Funder Increasing the impact of funders eg, effective commissioning Sector NPC works at the nexus between charities and funders Consultancy Think tank Strengthening the partnership eg, collaboration towards shared goals 2
  3. 3. COUNTERFACTUAL • Comparing the world with your organisation in it with what the world would be like without it. 3
  4. 4. WHY MEASURING IMPACT IS IMPORTANT Raises profile Motivates staff The Brandonsecure provides Helps Centre Influences the counselling funding and psychotherapy to debate on young people between the ages of 12 “what works” and 21 Improved services Taken from: Rickey, B, Lumley, T and Ni Ogain, E . (2011) A Journey to Greater Impact New Philanthropy Capital. 4
  5. 5. THEORY OF CHANGE • Links activities intermediate outcomes  final outcomes – clarifies what the activities aim to achieve and how – provides a structure for identifying what can be measured – provides the case for why achieving intermediate outcomes is important A conceptual map of how activities lead to outcomes 5
  6. 6. EXAMPLE THEORY OF CHANGE PARENTS’ COUNSELLING SERVICE Counselling Clients‟ emotional or psychological difficulties decrease Clients‟ capacity for self care increases Clients‟ ability to support their children's healthy development improves Activities Intermediate Outcomes Parent / Child interaction improves Children‟s emotional resourcefulness improves Final outcomes 6
  7. 7. WHAT: BEFORE YOU THINK ABOUT COLLECTING DATA ASK YOURSELF: Do you already collect this data? Yes Use this data (if it is fit for purpose) No Has anyone already No proved the causal link you want to make? Yes Is it really important you need data on this outcome? Yes Does anyone else collect this data? No Use research to limit your data collection Yes Don‟t collect unless you decide it is really important Yes Can you access this data? No No Develop your own data source 7
  8. 8. LEVELS OF EVIDENCE Credibility Anecdotes / quotes Case Self-reported studies change Basic Before and after survey Control groups Randomised control trial Advanced 8
  9. 9. LOTS OF DECISIONS • Budget • Stakeholders • Timing • Participants • Tools 9
  10. 10. QUANTITATIVE AND QUALITATIVE DATA • Quantitative data (numbers) – Statistical estimates – Prevalence of views, attitudes and experiences How many people find our newsletter useful? – Admin data/ questionnaires (paper, web, etc.) • Qualitative data (words) – Detailed understanding – In depth interviewing (telephone/ face to face) What factors influence people‟s views of our newsletter? 10
  11. 11. WHEN TO MEASURE? • Pre-post design: Before measure Programme / service After measure • What attribution can you claim? • Perception of beneficiaries and those delivering the programme: important but subjective 11
  12. 12. COMPARISON GROUPS Intervention group Before measure Programme / service After measure Comparison group Before measure After measure • Compare before and after data for participants and nonparticipants, ideally matching on their key characteristics 12
  13. 13. ALTERNATIVES Comparison groups • Waiting lists • Dosage: users who‟ve received a smaller „dose‟ of a service • Type of service: comparing users who‟ve received one type of service compared to another No comparison group • Combine pre and post data with respondents‟ perspectives • Collect data on other events and changes in circumstances which might have contributed 13
  14. 14. 5 TIPS FOR MEASURING YOUR IMPACT 1. Decide what‟s important 2. Keep it simple 3. Know your limits 4. Consider investing in training 5. Think about the future 14