Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Operationalisng SROI


Published on

Lessons from researching the impact of advice services

  • Be the first to comment

  • Be the first to like this

Operationalisng SROI

  1. 1. Operationalising Social Return on Investment in advice services Michelle Farr¹, Nick Abercrombie¹, Beth Jaynes¹, Sue Milner² and Peter Cressey¹ Department of Social and Policy Sciences¹, Politics, Languages & International Studies (PoLIS)² University of Bath
  2. 2. Operationalising SROI How can social return on investment be used to value advice services? 1. Short overview of the process - Michelle • From client recruitment to interviewing to analysis to SROI 2. Applying SROI - Nick • What are the issues and difficulties associated with applying SROI? 3. How our research tools are currently being adapted and developed for a range of advice services – Beth • Current project developing these methods for a range of advice organisations
  3. 3. What is social return on investment? • Outcome based evaluation • It seeks to measure changes that are most important to different stakeholders • Form of cost-benefit analysis • takes into account economic, social and environmental benefits • Intends to create a ratio of benefits to investment SROI= Value of benefits Investment
  4. 4. Data collection in a SROI • Focussing on outcomes • Changes that have occurred as a result of CAB • Mapping the impact of advice through interviews – who has been affected by the changes and how? • Deadweight • What would have happened without the client getting advice from the CAB? • Would they have gone to another advice service or got the same support elsewhere? • Would the same outcomes have occurred? • Benchmarks often used – what would have happened without the CABs intervention?
  5. 5. Data collection in a SROI • Attribution • How much credit can the CAB take for the outcomes? (other organisations, client action) • Expressed as a percentage – not exact science! • Displacement • How much does the outcome displace other outcomes? • Benefit period and drop-off • How long is the benefit of the outcome likely to last for? • Valuation of outcomes How do we attach a financial value to the outcomes?
  6. 6. Research tools • Speaking to clients about the outcomes of advice • Recruitment sheet, topic guides, asking clients about attribution, deadweight (Appendix 1) • Analysis, developing financial values and proxies (Appendix 2) • Developing rules around attribution, deadweight, and drop-off (Appendix 4) • Creating 80 impact maps (e.g. Table 3.1) • Range of different outcomes (e.g. Table 3.5) • Great sensitivity of SROI ratios, therefore did sensitivity analysis (Appendix 5).
  7. 7. Problems, problems… (Nick) 1. The problem of casework; a problem of cost. 2. The problem of attribution, deadweight; a problem of acceptance. 3. The problem of stakeholders; a problem of point of view.
  8. 8. The problem of casework 1. CAB offers a wide range of advice to anyone. 2. A very large number of outcomes are possible. Petra has 700. We logged 240 outcomes from advice and outcomes will have different values. 3. Every client’s combination of outcomes is unique. So each client has to have an impact map all to themselves. Very demanding research problem. 4. Very costly. 3 to 4 times the actual grant. Only possible with a lot of input – often expert – from volunteers. 5. Solution is either for the whole advice sector to rely on a few studies or to develop modal impact maps. 6. True not only of advice but most casework agencies.
  9. 9. The problems of attribution and deadweight 1. Attribution, deadweight, displacement, drop-off. 2. Ratio very sensitive to variations in these. Attribution or deadweight as an example. 3. Tried very hard in interviews to mitigate. 4. RCTs the traditional solution. Government policy. A problem of acceptance. RCT+CB in education. 5. RCTs hard work in these circumstances. Why?
  10. 10. The problem of stakeholders 1. SROI=CB + stakeholders. CB usually an investment tool. But need to consider all who my be affected by an intervention not just the investors. Hence social ROI. 2. But how to restrict the idea of stakeholder. Pebble in the lake. Drawing the line may be arbitrary. 3. We treated the client as primary stakeholder. So largely from the client’s point of view. But family impacts to some extent plus state agencies. 4. The gain from one stakeholder’s POV may be a loss for another e.g. benefit or debt 5. Our solution is to present an SROI divided into different types of value for different stakeholders.
  11. 11. Adapting Research Tools (Beth) Advice Partnership for Excellence (APEX) • Advice Services Transition Fund (ASTF) project, funded by the Big Lottery • One of the objectives of the funding is to measure impact • Using SROI methodology we will measure the impact of advice organisations and one council department • 100 service users • Baseline data collected using GHQ12 • 2 interviews over a year • Dependent on volunteer time
  12. 12. Anticipating Challenges • Co-ordinating project over a number of organisations will be difficult • Different clients journeys - CAB - often outcomes take a while but for others they are immediate • Criteria of some organisations - only some people are allowed access and guaranteed success - how do we allow for this? • Need to have slightly different topic guide for each organisation but keep most relevant questions the same
  13. 13. Ways forward • Can a survey based on a range of quantitative and qualitative questions be developed to cut down on transcribing costs and time? • Using more fine tuned research questionnaires • Moving from short Warwick and Edinburgh Mental Well Being Scale (WEMWBS) to General Health Questionnaire (GHQ12) • Developing modal impact maps from our existing data