Operationalising Social Return on
Investment in advice services
Michelle Farr¹, Nick Abercrombie¹, Beth Jaynes¹, Sue
Milner² and Peter Cressey¹
Department of Social and Policy Sciences¹,
Politics, Languages & International Studies (PoLIS)²
University of Bath
How can social return on investment be used to value
1. Short overview of the process - Michelle
• From client recruitment to interviewing to analysis to
2. Applying SROI - Nick
• What are the issues and difficulties associated with
3. How our research tools are currently being adapted
and developed for a range of advice services – Beth
• Current project developing these methods for a
range of advice organisations
What is social return on investment?
• Outcome based evaluation
• It seeks to measure changes that are most
important to different stakeholders
• Form of cost-benefit analysis
• takes into account economic, social and
• Intends to create a ratio of benefits to
SROI= Value of benefits
Data collection in a SROI
• Focussing on outcomes
• Changes that have occurred as a result of CAB
• Mapping the impact of advice through interviews –
who has been affected by the changes and how?
• What would have happened without the client getting
advice from the CAB?
• Would they have gone to another advice service or
got the same support elsewhere?
• Would the same outcomes have occurred?
• Benchmarks often used – what would have
happened without the CABs intervention?
Data collection in a SROI
• How much credit can the CAB take for the outcomes?
(other organisations, client action)
• Expressed as a percentage – not exact science!
• How much does the outcome displace other outcomes?
• Benefit period and drop-off
• How long is the benefit of the outcome likely to last for?
• Valuation of outcomes
How do we attach a financial value to the outcomes?
• Speaking to clients about the outcomes of advice
• Recruitment sheet, topic guides, asking clients about attribution,
deadweight (Appendix 1)
• Analysis, developing financial values and proxies
• Developing rules around attribution, deadweight, and
drop-off (Appendix 4)
• Creating 80 impact maps (e.g. Table 3.1)
• Range of different outcomes (e.g. Table 3.5)
• Great sensitivity of SROI ratios, therefore did sensitivity
analysis (Appendix 5).
Problems, problems… (Nick)
1. The problem of casework; a problem of cost.
2. The problem of attribution, deadweight; a problem of
3. The problem of stakeholders; a problem of point of
The problem of casework
1. CAB offers a wide range of advice to anyone.
2. A very large number of outcomes are possible. Petra has
700. We logged 240 outcomes from advice and outcomes
will have different values.
3. Every client’s combination of outcomes is unique. So each
client has to have an impact map all to themselves. Very
demanding research problem.
4. Very costly. 3 to 4 times the actual grant. Only possible
with a lot of input – often expert – from volunteers.
5. Solution is either for the whole advice sector to rely on a
few studies or to develop modal impact maps.
6. True not only of advice but most casework agencies.
The problems of attribution and deadweight
1. Attribution, deadweight, displacement, drop-off.
2. Ratio very sensitive to variations in these. Attribution or
deadweight as an example.
3. Tried very hard in interviews to mitigate.
4. RCTs the traditional solution. Government policy. A
problem of acceptance. RCT+CB in education.
5. RCTs hard work in these circumstances. Why?
The problem of stakeholders
1. SROI=CB + stakeholders. CB usually an investment tool.
But need to consider all who my be affected by an
intervention not just the investors. Hence social ROI.
2. But how to restrict the idea of stakeholder. Pebble in the
lake. Drawing the line may be arbitrary.
3. We treated the client as primary stakeholder. So largely from
the client’s point of view. But family impacts to some extent
plus state agencies.
4. The gain from one stakeholder’s POV may be a loss for
another e.g. benefit or debt
5. Our solution is to present an SROI divided into different types
of value for different stakeholders.
Adapting Research Tools (Beth)
Advice Partnership for Excellence (APEX)
• Advice Services Transition Fund (ASTF) project, funded
by the Big Lottery
• One of the objectives of the funding is to measure
• Using SROI methodology we will measure the impact of
advice organisations and one council department
• 100 service users
• Baseline data collected using GHQ12
• 2 interviews over a year
• Dependent on volunteer time
• Co-ordinating project over a number of organisations
will be difficult
• Different clients journeys - CAB - often outcomes take a
while but for others they are immediate
• Criteria of some organisations - only some people are
allowed access and guaranteed success - how do we
allow for this?
• Need to have slightly different topic guide for each
organisation but keep most relevant questions the same
• Can a survey based on a range of quantitative and
qualitative questions be developed to cut down on
transcribing costs and time?
• Using more fine tuned research questionnaires
• Moving from short Warwick and Edinburgh Mental Well Being
Scale (WEMWBS) to General Health Questionnaire (GHQ12)
• Developing modal impact maps from our existing data