Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lena Etuk Why Measure Social Impact?

40 views

Published on

Presented to the NSW Marketing Working Group Seminar on Impact Measurement, November, 2017

Published in: Education
  • Be the first to comment

  • Be the first to like this

Lena Etuk Why Measure Social Impact?

  1. 1. Why Measure Social Impact? 7 November, 2017 Presentation for The State Library of NSW Lena Etuk, CSI Senior Research Officer
  2. 2. 2 AGENDA Introduction to the Centre for Social Impact Key Terms Why measure social impact? What to consider for your journey into outcomes and impact measurement
  3. 3. 3 THE CENTRE FOR SOCIAL IMPACT Our purpose is to catalyse positive social change. We do this by engaging and working with people, communities, and organisations to grow their capabilities through research, education, and leadership development.
  4. 4. 4 AGENDA Introduction to the Centre for Social Impact Key Terms Why measure social impact? What to consider for your journey into outcomes and impact measurement
  5. 5. 5 SOME FREQUENTLY USED WORDS AND PHRASES • Logic Models • Inputs • Outputs • Outcomes • Impact • Impact assessment • Outcome measurement • Evaluation • Monitoring
  6. 6. 6 LOGIC MODEL Inputs Provided resources for an intervention; e.g. budget for education Activities E.g., construction of schools or hiring additional teachers Outputs Technical results of an intervention; e.g. number of new schools Outcomes Direct effects of an intervention; increase of school enrolment Impact Wider effects; e.g. contribution to poverty reduction because of improved educational outcomes Your Planned Work Your Intended Results If…then… If…then… If…then… If…then…
  7. 7. 7 OUTPUTS The direct products of program activities and may include types, levels and targets of services to be delivered. (Kellogg, 2004) • Size and/or scope of the services and products delivered or produced • Number of services or products delivered • Participation rates • Hours of service provided For example: • Increased community access and participation in health promotion and primary care services • Evaluation and measurement program in place • # of workshops attended
  8. 8. 8 OUTCOMES Can be both the expected or unexpected effects of a program, initiative or activity. Outcomes • Knowledge • Attitudes • Values • Behaviours • Conditions Different stakeholders • Clients • Families • Communities • Funders • Government • Society • Environment Positive Negative Can be… Short-term Medium-term Long-term Occur at different times
  9. 9. 9 Activity: Which of these are outputs and which are outcomes? Outputs Outcomes Number of audits completed 25 people attend awareness workshop Visits to community centre / service provider 100 people access information line Reduced social isolation Improvement in family life & well-being Increased level of confidence 20% improvement on knowledge scale OUTPUTS VS. OUTCOMES
  10. 10. 10 SOCIAL IMPACT Not clearly or consistently defined: • Value created (Mulgan,2010) • The broad or long-term effects of a project or organisation’s outputs, outcomes, and activities. (New Philanthropy Capital, 2011) • The portion of an outcome change that can be attributed uniquely to a program, that is, with the influence of other sources controlled or removed. (Rossi, Lipsey and Freeman, 2004: 431) • Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. (OECD, 2010:25)
  11. 11. 11 • Involves identifying how desired outcomes should be measured • And involves collecting outcomes data • Largely quantitative in nature • Measures the prevalence of goal attributes among beneficiaries Domains Outcomes Indicators Measures “75% of our program participants are currently satisfied with their health” “On average, our program participants had a moderate level of knowledge about financial products, now they have a high level” OUTCOMES MEASUREMENT
  12. 12. 12 “The systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results.” (OECD Development Assistance Committee, 2010:21) • Process Evaluation // Outcome Evaluation // Impact Assessment • Formative Evaluation // Summative Evaluation Evaluation is a process rather than a specific method or approach. Monitoring vs Evaluation • Monitoring is on-going collection of data • Evaluation is analysis of data to inform judgement EVALUATION
  13. 13. 13 QUESTIONS ABOUT TERMS?
  14. 14. 14 AGENDA Introduction to the Centre for Social Impact Key Terms Why measure social impact? What to consider for your journey into outcomes and impact measurement
  15. 15. 15 WHY MEASURE OUTCOMES & IMPACT? We need to measure to understand if we are making a difference. • Are people really any better off? • Are people more resilient, included and connected? • Have our actions had any impact? Counting numbers of people and amount of activity alone aren’t perfect indicators of results. Why else might we measure outcomes and impact?
  16. 16. 16 WHY MEASURE OUTCOMES & IMPACT? Other benefits: • Learning, development • Communicating and branding • Accountability and compliance • Increased efficiency • Organisational sustainability Before we invest millions, maybe we should test it and see if it works
  17. 17. 17 THE OTHER FACTORS THAT MAY MOTIVATE US • Performance management & improvement • Competitive market: informed consumers, facilitated choice • Enable meaningful comparisons between programs & orgs
  18. 18. 18 WHY IS IT IMPORTANT? A consistent approach to measuring outcomes and impact will help to: • Clarify outcomes and results • Inform decision making • Help to work with others to address ‘wicked’ problems
  19. 19. 19 QUESTIONS ABOUT WHY WE SHOULD MEASURE IMPACT?
  20. 20. 20 AGENDA Introduction to the Centre for Social Impact Key Terms Why measure social impact? What to consider for your journey into outcomes and impact measurement
  21. 21. 21 THE OUTCOME & IMPACT MEASUREMENT CYCLE 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  22. 22. 22 UNDERSTANDING PHASE • Clarify the measurement needs of the organisation or program – Information, evidence, questions – Reflect on the role of context • Consider existing external and internal systems, frameworks, and reporting requirements • Consider who needs to be involved in measurement, and when 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  23. 23. 23 PROGRAM MODEL PHASE • Clarify the rationale for the program, and assumptions – Theory of Change • Articulate your 3 Ps: – Purpose – Process – Progress • Create a Logic Model (inputs, activities, outputs, outcomes, impact) 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review Progress: to what extent have we achieved our purpose/made a difference? Process: how are we going to achieve it? Purpose: what are we trying to achieve?
  24. 24. 24 OUTCOME FRAMEWORK PHASE • Clarify what the outcomes and impacts in your Logic Model really mean • Identify and select good indicators – Measurable markers that demonstrate an outcome • Identify and select valid measures – The direct instruments that ascertain the existence of something 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  25. 25. 25 DATA COLLECTION PLAN PHASE • Only plan to collect what you will use • Choose your data collection methods – Outcomes can be measured in a variety of ways, particularly when measured in the context of outcomes evaluation – Most common: • Questionnaires or surveys designed in house • Interviews • Case studies 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  26. 26. 26 ANALYSIS & COMMUNICATION PLAN PHASE • Establish how to approach the analysis of data • Identify appropriate channels for communication and engagement 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  27. 27. 27 IMPLEMENTATION PLAN PHASE • A plan for how your organisation will roll-out outcome and impact measurement – Pilot vs. across whole organisation – Capacity-building among existing staff vs. hiring new staff w/expertise vs. procuring consultants – Any work needed to improve the measurement or learning culture of the organisation 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation w/continual monitoring & review
  28. 28. 28 IMPLEMENTATION PHASE • Measure the outcomes and impacts • Communicate findings • Adjust measurement activities, if needed 1. Understanding 2. Program Model 3. Outcome Measurement Framework 4. Data collection plan 5. Analysis and communication plan 6. Implementation Plan Implementation
  29. 29. 29 QUESTIONS ABOUT THE IMPACT MEASUREMENT JOURNEY?
  30. 30. 30 REFLECTION QUESTIONS • Are the expectations for outcome measurement understood? Internal and external • Do your organisations support failure and learning? Are supports and expectations in place? • What would it take to develop a common language and understanding of the key terms for outcome measurement across your organisations? • Where do you think your organisations are in regards to readiness for outcomes measurement?
  31. 31. 31 AMPLIFY SOCIAL IMPACT, A FUTURE RESOURCE FROM CSI Tech platform to answer critical questions: 1. Where are social problems acute and amongst which populations? 2. How do we know if we’re impacting the problem? 3. Are our strategies and interventions working and worthy of replication? = = = + Social Progress Index
  32. 32. 32 CONTACT DETAILS Lena Etuk Senior Research Officer P +61 2 8936 0929 E l.etuk@unsw.edu.au @CSIsocialimpact www.csi.edu.au

×