Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring the Impact and ROI of community/social investment programs

Getting better at understanding community/social investment programs and strategies. What works - how it works and why it works?

Related Books

Free with a 30 day trial from Scribd

See all
  • Login to see the comments

  • Be the first to like this

Measuring the Impact and ROI of community/social investment programs

  1. 1. Measuring the social impact and return on investment of development programmes Reana Rossouw Next Generation Consultants
  2. 2. Introduction / Basics 2016/04/15Next Generation Consultants 2
  3. 3. Definitions What is impact? Measures ‘the difference being made by an intervention’ Impact measurement is the process of providing evidence that your organisation/program - is doing something that provides a real and tangible benefit. What is return? Measures ‘the funder benefit/gain’ What is an impact measurement framework? The activities an organisation carries out can have long-term effects on beneficiaries, beneficiaries’ families and the broader community. An impact measurement framework seeks to identify and quantify/qualify this impact/change. An impact measurement framework provides the structure for assessing all aspects of impact. 2016/04/15Next Generation Consultants 3
  4. 4. Why is measuring impact and return important? Measuring impact can help to Build on the things you are doing well and to learn from the challenges you have faced/experienced. This feeds into good practice and means the organisation will learn and improve continuously. Just as financial accounts prove the viability of a business, impact measurement can show a robust and rigorous approach to providing community or economic or environmental benefits. 2016/04/15Next Generation Consultants 4
  5. 5. Why the pressure to measure? The debate of impact and return on investment are playing out in three arenas: In private/corporate foundations and CSI divisions Aiming to be more strategic about their social/community investments and development programmes In nonprofit organisations in response to pressures from investors, corporates, foundations and government To be more accountable for the resources received and programme outcomes expected Among all other role-players such as government agencies, intermediary organisations, and international development agencies Seeking to improve development effectiveness and lessen dependency on development aid 2016/04/15Next Generation Consultants 5
  6. 6. Why now? Funders are increasingly asking for verifiable results – to understand the difference they make, directly and indirectly in the economy, society and the environment This trend is accelerating, and the sector is increasingly looking to pay for/by results – and to learn from what they, and those they fund, do It is not just donors that care about impact. In a age of competition, transparency, and recessionary economies there is a growing competition for resources 2016/04/15Next Generation Consultants 6
  7. 7. Impact Assessment - Variety of purposes One can and should use impact data to make programme decisions across investment and development portfolios One can and should use impact data to make funding decisions within investment and development portfolios It is about building a unifying measurement standard, as well as a conceptual framework for understanding the biggest impact across a Rand value unit. So - it is not just about comparing one investment or one development portfolio to another or one organisation to another, but to also determine which programme/organisation/investment yields the highest return/impact for the most effective use of resources Therefore, we require a combination of the breadth of quantitative with the depth of qualitative evaluation/assessment methods 2016/04/15Next Generation Consultants 7
  8. 8. Differences Definition Detail MONITORING: Regular systematic collection and analysis of information to track the progress of programme implementation against pre-set targets and objectives. DID WE DELIVER? • Clarifies programme objectives • Links activities and their resources to objectives • Translates objectives into performance indicators and sets targets • Routinely collects data on these indicators and compares actual results with targets • Reports progress to managers and alerts them to problems EVALUATION: Objective assessment of an on-going or recently completed project, programme or policy, its design, implementation and results. WHAT HAS HAPPENED AS A RESULT? • Analyses why intended results were or were not achieved • Assesses specific casual contributions of activities to results • Examines unintended results • Provides lessons, highlights significant accomplishments or programme potential and offers recommendations for improvement IMPACT ASSESSMENT: Assesses what has happened as a result of the intervention and what may have happened without it, from a future point of time. HAVE WE MADE A DIFFERENCE AND ACHIEVED OUR GOAL? • Seeks to capture and isolate the outcomes that are attributable (or caused by) the programme • Will review all fore-going monitoring and evaluation activities, processes, reports and analysis • Provides an in-depth understanding of the various causal relationships and the mechanisms through which they operate • May seek to synthesize, compare, contrast a range of interventions in a region, timeframe, sector or reform area 8
  9. 9. The impact value chain 2016/04/15Next Generation Consultants 9 Input Resources that are deployed in service of a certain (set of) activities Cash, Products, Services, Hours, Non-cash – buildings, time, books, resources (wheelchairs, books) Invested resources or capital deployed Activity Actions, or tasks that are performed in support of specific impact objectives Actions by an organisation to facilitate a change Activities undertaken to deliver on program objectives goals Output Tangible, immediate practices, products and services that result from the activities that are undertaken Number of beneficiaries served by an organisation/ program Services rendered through the capital provided Outcome Changes, or effects on individuals/commu nities that follow from the delivery of products and services Changes among beneficiaries i.e. increased capacity to earn a living Impact Changes or effects on society or the environment that follow from outcomes that have been achieved Changes in broader environment for instance reduced incidences of crime Impact on society due to capital deployed Return Benefit gained/achieved by the funder as a result of the input resources Changes in the business or operating environment for instance better stakeholder relationships Benefit by the funder due to the impact capital invested
  10. 10. Guidelines/process for impact measurement Goal setting Framework development and indicator selection Data Collection Data Validation Data Analysis Data Reporting Data Driven Management Set goals Articulate the desired impact of the investment – establish a theory of change/value creation process to form the basis of strategic planning and decision-making to serve as a reference point for investment performance Develop Framework and Select indicators Determine metrics/indicatos to be used for assessing the performance of the investment - the framework integrates indicators and outlines how specific data are captured and used, it uses indicators that align with existing standards to measure progress/impact Collect data Ensure that the information, technology, tools and resources, human capital and methods used to obtain and track data for the anticipated impact are effectively utilised Validate Data Validate data to ensure sufficient quality – verify that impact data is complete and transparent by checking assumptions and calculations against known data sources, where applicable (baselines) Analyse Data Distil insights from the data collected – review and analyse data to understand how investments are progressing against impact goals/objectives Report Data Share progress with stakeholders – distribute impact data coherently, credibly, and reliably to effectively inform decisions by all stakeholders Make Data-driven management decisions Identify and implement mechanisms to strengthen the investment/development process and outcomes – assess stakeholder feedback on reported data and address recommendations to make changes to the investment or Theory of Change for sustainable development 2016/04/15Next Generation Consultants 10
  11. 11. Indicators Indicators define/measure and demonstrate the outcome of an objective Indicators should be easily measurable (within reason) and meaningful to stakeholders In selecting indicators the following factors may be influential: Who — the target population to be measured/assessed How much — the degree of change that is expected How many — the amount of change among the target population that would indicate a successful level of achievement When — the time frame in which this change should occur Indicators can be quantitative or qualitative or both Quantitative indicators measure changes expressed as numbers or statistics, e.g. frequency of behaviours, number of attendees Quantitative data can also measure changes in attitudes, beliefs, and perceptions when respondents express opinions in the form of ratings or rankings An advantage of a quantitative data collection approach is that data can be easier to collect and analyze than qualitative information. However, data that is easy to gather may not be the most illuminating or insightful Qualitative indicators relies on observations of changes with the intent of understanding patterns and relationships. Data are often in the form of descriptions, narratives, and open-ended responses Qualitative data might be used to capture individual and collective stories of social change, including personal perceptions Sometimes qualitative evidence is the only way to document changes in relationships and contexts 2016/04/15Next Generation Consultants 11
  12. 12. Important indicator selection issues Involve stakeholders in defining indicators Consider both quantitative and qualitative indicators What is most important to a program or initiative may not be easiest thing to measure. In collecting data that is easily accessible or purely quantitative, stakeholders run the risk of being dismissive of the true and deeper impact of a program Conversely, a cursory anecdotal review of a program done by those who are predisposed to view it as successful would not likely hold up to scrutiny from outsiders Determine if your outcomes—and therefore the indicators you define—are short-, intermediate- or long-term Measurement of short-term change will most likely focus on the implementation of a project (How did it work? How effective was the design?) or on establishing a baseline from which to assess change down the road. Intermediate change can be measured by defining benchmarks. A benchmark is a type of indicator that refers to the level of change an organisation expects to make from its baseline. Most evaluators and researchers agree that impact takes time to occur and that outcomes are only measurable in the intermediate- or long-term. The longer a program runs, the more complex its selection of indicators and the more likely, or even necessary, it becomes to use benchmarks.2016/04/15Next Generation Consultants 12
  13. 13. Important indicator selection issues Consider a program’s context and its complexity as this will affect results and the selection of indicators. Select indicators at the beginning of a program or project in order to plan for data collection, but also allow for indicators to be defined or revised as new outcomes emerge during program implementation. Decide on a modest range of realistic indicators. It is nearly impossible for a few indicators to capture complex social change, particularly since change tends to happen incrementally and can be attributed to numerous inputs. Avoid common pitfalls. One frequent mistake is that people choose outcomes that they believe they can easily achieve and measure based on the desire to show success. Conversely, they may ignore outcomes that are difficult to achieve and measure. Another mistake is to disregard negative outcomes or not be alert to unexpected outcomes that, if recorded and assessed, can be just as important in terms of understanding impact. Yet another pitfall is the tendency to attribute causality for social change to one modest program when that change is caused by multiple factors. 2016/04/15Next Generation Consultants 13
  14. 14. What Impact Assessment is all about Impact Identify, interpret, improve, investigate, involve, inform To provide evidence To demonstrate performance To prove accountability To show programme/investment effectiveness To demonstrate shared/blended value To empower and capacitate all stakeholders Ultimately - to alleviate, reduce and eradicate poverty and contribute to sustainable development 2016/04/15Next Generation Consultants 14
  15. 15. Conducting an impact assessment Methodology: Impact Investment Index™ Developed by Next Generation Consultants 2016/04/15Next Generation Consultants 15
  16. 16. Impact Assessment – WHAT? 2016/04/15Next Generation Consultants 16
  17. 17. Calculating Impact – The Process 2016/04/15Next Generation Consultants 17 Information Sources (Strategies, applications, contracts, evaluation reports, site inspections, engagement) Primary, secondary and tertiary assessment Data score sheet – identifying and calculating impact and return Impact per stakeholder (QL & QN) Return for investor Dimension of impact and return Analysis, interpretation, triangulation of data Impact per programme, per focus area per stakeholder group Return per programme per focus area Cost benefit analysis Shared Value X:Y Recommendation strategic, operational and programmatic
  18. 18. How much was spent – where – on what? 2016/04/15Next Generation Consultants 18
  19. 19. Case Study 1 - Multichoice Total Impact – Per programme Total Impact – Per Focus Area 2016/04/15Next Generation Consultants 19
  20. 20. Case Study 1: Multichoice Total Return per Brand Total Return by Investment 2016/04/15Next Generation Consultants 20
  21. 21. Case Study 2: Pioneer Foods Spend per focus area Impacts per focus area 2016/04/15Next Generation Consultants 21
  22. 22. Case Study 2: Pioneer Foods Impact across the triple bottom line 4 3 3 3 3 3 3 3 2 2 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 Economic Impact 7 5 4 4 0 1 2 3 4 5 6 7 8 F&TFA- Limani Aqua Noir Heart MOT Social Impact 2016/04/15Next Generation Consultants 22 8 8 8 3 2 2 0 1 2 3 4 5 6 7 8 9 Environmental Impact
  23. 23. Case Study 2: Pioneer Foods Impact over time 6 3 2 2 2 2 2 0 1 2 3 4 5 6 7 Short-Term Impact 4 3 2 2 2 2 2 0 0,5 1 1,5 2 2,5 3 3,5 4 4,5 Medium Term Impact 2016/04/15Next Generation Consultants 23 7 4 3 3 3 3 3 0 1 2 3 4 5 6 7 8 Long-Term Impact
  24. 24. Case Study 2: Impact per focus area 97 91 Paardeberg WWF Impact Scores for SED Focus Area - Environment 100 97 91 79 78 60 60 50 38 Impact Scores for SED Focus Area - Food Security 2016/04/15Next Generation Consultants 24 103 61 MOT PFECT Bursary Scheme Impact Scores for PFECT Focus Area - Education 71 25 Aqua Noir Katmakoep Impact Scores for Focus Area - Enterprise Development
  25. 25. Case Study 3 – BHP Billiton Metalloys – Programme performance Top five 94 76 69 64 62 Bottom five 34 44 46 46 47 2016/04/15Next Generation Consultants 25 60 54 52 52 Middle four
  26. 26. Case Study 3: BHP Billiton Metalloys - Impact across focus areas per programme Commuity Impact High Developmental Business Impact High Strategic HIV and Me 94:7 Community Impact Low Charitable Inkululeko 52:1 Cansa 44:2 Reach for a Dream 34:3 Business Impact Low Commercial Health Average 56:3 Community Impact High Developmental Eureka School 76:2 Meyerton High School 69:1 Business Impact High Strategic Dr Malan School 64:3 PC Literacy for Educators 52:4 Community Impact Low Charitable Springfield Primary 46:1 Sibongile School 46:2 Business Impact Low Commercial Wings for Life 54:3 CASME 47:3 Education Average 57:2 2016/04/15Next Generation Consultants 26 Commuity Impact High Developmental Business Impact High Strategic Kotulong Community Centre 62:4 Raizcorp Business Incubation 60:4 Community Impact Low Charitable MCD Training 52:3 Business Impact Low Commercial Poverty Alleviation Average 58:4
  27. 27. Impact vs. Return: Programmes vs. Focus Areas Commuity Impact Above Average Eureka School 76:2 Meyerton High School 69:1 Dr Malan School 64:3 Business Impact Above Average HIV & ME 94:7 Kotulong CC 62:4 Raizcorp Business Incubation 60:4 Community Impact Below Average Wings for life 54:3 MCD Training Centre 52:3 Adv. of Science and Maths 47:3 Reach for a Dream 34:3 Sibonile School 46:2 CANSA 44:2 Inkululeko Project 52:1 Springfield Primary 46:1 Business Impact Below Average PC Literacy for Educators 52:4 Total Impact 852 Total Return 43 Overall Averages 56:3 Commuity Impact Above Average Business Impact Above Average Education 454:19 Community Impact Below Average Health 224: 13 Poverty Alleviation 174:11 Business Impact Below Average Total Impact 852 Total Return 43 Overall Averages 284:14 2016/04/15Next Generation Consultants 27
  28. 28. Case Study 3: BHP Billiton Metalloys – Impact vs. Cost 2016/04/15Next Generation Consultants 28 Individual cost per impact and return on investment for the community investment and development programmes Total Spend Programmes Total Impact Total Return R 62 678 764,00 15 852 43 Cost per Impact R 4 933. 39 Cost per Return R 78 841. 21 Individual cost per impact per Focus Area Focus Area Spend Programmes Total Impacts Cost per Impact Health R 1 364 500.00 3 217 R 2 096.00 Education R 36 314 264.10 8 472 R 9 617.00 Poverty Alleviation R 25 000 000.00 3 158 R 52 742.60 Individual cost per return per Focus Area Focus Area Spend Programmes Total Returns Cost per Return Health R 1 364 500.00 3 15 R 30 322.22 Education R 36 314 264.10 8 27 R 168 121.59 Poverty Alleviation R 25 000 000.00 3 11 R 757 575.75 The table below indicates that the Poverty Alleviation portfolio/focus area delivers the least community impact, whilst the health and educational portfolios delivered more cost effective impact for communities. The table below highlights the fact that to achieve a single impact in the community R4 933 needs to be spend per programme. Similarly, R78 800 needs to be spent to achieve a single return on investment for BHP Billiton Metalloys South Africa. The table below clearly indicates that the health portfolio not only had the highest return on investment for BHP Billiton Metalloys, but was also the most cost effective portfolio. The Educational portfolio delivered the second highest return on investment and the Poverty Alleviation was not only the most expensive, had the least community impact, but also the least return on investment.
  29. 29. Return on Investment 2016/04/15Next Generation Consultants 29
  30. 30. Business value of determining ROI Knowledge Deep understanding of value and impact as well as risks Comparative data – industry/sector Insights into and across impact dimensions Insights into stakeholder groups affected Action New or enhanced business decisions, practices and behaviours Develop new products/services/markets Changes policies, strategies and practices to increase impact and return Report in a more credible, integrated and useful way Results Improved performance – profitability/competitiveness Reduce potential risks – community activism/licencing Save costs – of court cases/mitigation of risks Enhanced stakeholder relationships Improved licence to operate conditions Improve trust and transparency 2016/04/15Next Generation Consultants 30
  31. 31. Return on investment – Indicators Strategic Aspects Support of corporate values and strategies Support of sustainability strategy/programs Support of future growth, development and market access Investor / Shareholder Aspects Share price not affected when industry or sector are targeted by activists Rated as industry leader in Sustainability Indices Increased investment from socially responsible investment funds Inclusion and high ratings in awards programs Reputation Aspects Recognition/awards Media coverage Increased brand awareness 2016/04/15Next Generation Consultants 31
  32. 32. ROI Impact Profit Aspects Sales generated from programs linked to products Value of new products and services generated from CI/CSI programs Increased worker productivity Increased share price (e.g. from attention of socially-screened investment funds) Environmental Aspects Costs mitigated from rehabilitation Costs saved from waste management/recycling Carbon emissions sequestrated Costs of fines Sector Specific Aspects Financial Sector • Economic trends and demographics and expanding workforce needs • Increasing regulatory activity (e.g. CRA, PRI, CRESA, JSE, investment screening) • Increasing equality/disparity between haves/have- nots – financial inclusivity • Globalization • Opportunities to brand company through community involvement Mining • Intensity of opposition • Previous negative incidents • Regulators’ sensitivities • Compatibility with existing development objectives • Reputation of company • Level of community involvement • Involvement of external advocates 2016/04/15Next Generation Consultants 32
  33. 33. ROI Impact Stakeholder Aspects Increased community/government awareness/positive relationships/stakeholder relations Decreased complaints/grievances/ activism/strikes/boycotts/negative press coverage Cost savings/avoidance Prevention of operational stoppages/delays Reducing/decreasing legal costs/law suits Support for market entry/expansion plans Savings Aspects Tax rebates received from philanthropic/ charity/social/community contributions Saved costs of free advertising space received from media coverage of the CI/CSI programs Legal fees averted (includes legal department staff time and projected billable hours from contracted firms) Savings Aspects Continue Crisis PR efforts averted (includes PR staff time and projected billable hours from contracted firms) Costs of avoided down-time from failure to receive building approval, work stoppages, etc. Reduced employee recruitment costs, reduced turnover costs, and/or reduced absenteeism Reduced employee training costs (e.g., through community service learning initiatives) Reduced customer turnover Other staff management hours saved 2016/04/15Next Generation Consultants 33
  34. 34. ROI Impact Customer Aspects Surveys indicating improved customer perceptions and impacts on shopping decisions Sales leads generated in specific geographic or demographic markets Development/increased sales of specific products/services in targeted geographic or demographic markets Annual brand tracking surveys indicating higher scores Collaboration/participation/co-design of new product/service development Greater participation/involvement/ contribution in community investment and development programs Increased brand awareness Increased customer acquisition/retention Operational Aspects Mitigation of operational risks (health/environment/safety) Support and enhancement of business operational requirements (integration, skills development, etc.) Compliance Aspects • B-BBEEE • Licence to operate • SLP Mandate/Strategy • DMR/King III/ICMM/IPIECA • Approval rates/new explorations/extensions • Rehabilitation • Drop in complaints/grievances • Global Compliance 2016/04/15Next Generation Consultants 34
  35. 35. ROI Impact Employee Aspects Positive response to utilizing volunteerism for professional development/skills development and team building Employee surveys demonstrating that volunteer activities contribute to leadership development Voted one of the best companies to work for Surveys showing increased employee morale from participation and increased numbers of employee volunteers, volunteer hours, and the number of company-sponsored volunteer projects Satisfaction surveys indicating positive impact and anecdotal evidence Employee training programs designed to use volunteers and products with most donations Employees learning to use products to that they are more equipped to sell/market them CSI/CI projects used for team building or during orientation/induction or other training Recruitment from communities where CSI/CI projects are run Internal surveys showing an increase in employee pride, morale and commitment as a result of employee involvement in volunteer activities Social Aspects Improvement of quality of life Community job creation / empowerment Improved stakeholder relations within the community Poverty reduction 2016/04/15Next Generation Consultants 35
  36. 36. IN CLOSING 2016/04/15Next Generation Consultants 36
  37. 37. 2016/04/15Next Generation Consultants 37 Impact: Thinking beyond evaluations: Decision •Now what? Synthesis •So what? Analysis •What impact was achieved? •What does the evidence show? Method •How will we determine impact or gather evidence? Information •How will we know? •What evidence do we need? Standard •What would indicate impact? Criteria for impact/ value of impact •What matters? Key Question •What do you want to know?
  38. 38. Thank You Reana Rossouw - Next Generation Consultants Next Generation Consultants are internationally recognized and have published extensively and spoken at local and international conferences. Copies of these articles, research papers, presentations, whitepapers and awards are available on: Website: Linkedin: generation-consultants Google+: rossouw Pinterest: Facebook: nextgenerationconsultants/ Slideshare: Please Note: The material is this presentation is copyrighted Permission must be obtained for using the material/content of this presentation. 2016/04/15Next Generation Consultants 38