This presentation assesses the delivery of well-being through the Rural Development Programme for England (RDPE) using a social impact measurement tool known as Social Return on Investment (SROI). It considers whether the goals of Axis 1 and 3 of the RDPE were fulfilled using this technique, and the use of SROI in future well-being measurement.
3. Rural development programmes
are policy tools capable of:
• providing incentives
• creating opportunities
• balancing development
• reducing inequality
• Improving wellbeing
Delivering Wellbeing
RDPE 2007-13
Axis 1: improving the
competitiveness of
agriculture and forestry
Axis 3: improving the
quality of life in
rural areas
5. SROI Outcomes
• Farm sustainability
• Carbon footprint
• Animal health
• Livestock skills
• Soil & land mgment
• Young generation
business start-up
Axis 3
SROI ratio 4:1
Axis 1
SROI ratio 1.5:1
• New enterprises
• Employment
opportunities
• Local facilities
• Service provision
• Social capital
• Natural & built
environment
6. Dimensions of well-being Axis 1 Axis 3
Material living (income, consumption,
wealth)
J K
Health K J
Education JJ JJ
Personal activities including work K J
Political voice L K
Social relationships K JJ
Insecurity J J
Environment JJ K
9. Dr John Powell
Senior Research Fellow
jpowell@glos.ac.uk
ccri.ac.uk
@CCRI_UK #CCRI2015
Editor's Notes
“…a positive physical, social and mental state;…
It requires that basic needs are met, that individuals have a sense of purpose, that they feel able to achieve important personal goals and participate in society.
It is enhanced by conditions that include supportive personal relationships, strong and inclusive communities, good health, financial and personal security, rewarding employment, and a healthy and attractive environment.
”DEFRA (2010) ‘Common understanding’ in Measuring Progress/Sustainability Indicators
A lot of work being done:
2009 report by Stiglitz and other economists sponsored by French Presidency
NEF developed working model of national accounts of well-being
Office of National Statistics
ONS concluded no clear definition but related to
Health,
Social relationships,
employment,
quality of the environment,
adequate income (financial security),
fairness and equality in society
Local factors are important to well-being (e.g. Access to green spaces and strength of community involvement).
Clearly complex – makes delivery difficult – require policy coordination and examination of how policies and programmes interact.
In order to deliver policies effectively need to be able to measure their outcomes and their impacts
________________________________________________________________
NEF report 2009 on a working model of National Accounts of Well-being.
The findings are compiled from data collected in a major 2006/2007 European cross-national survey through a detailed module of well-being questions, designed by the University of Cambridge, NEF (the new economics foundation) and other partners. This represents the most comprehensive dataset on subjective well-being for any nation to date.
Our working model is built on two headline measures which capture personal well-being and social well-being.
Personal well-being is broken down into five main components with a number of subcomponents: emotional well-being (positive feelings and absence of negative feelings); satisfying life; vitality; resilience and self-esteem (self-esteem, optimism and resilience); and positive functioning (which covers autonomy, competence, engagement, and meaning and purpose).
Social well-being is made up of two main components: supportive relationships, and trust and belonging. In addition to these indicators, an example of a well-being indicator within a specific area of people’s lives was also created – a satellite indicator of well-being at work. This measures job satisfaction, satisfaction with work-life balance, the emotional experience of work, and assessment of work conditions.
Stiglitz et al (Commission Report 2009)
Multi-dimensional definition needed for well-being. Key dimensions include:
- Material living standards (income, consumption, wealth)
- Health
- Education
All Party Parliamentary report on wellbeing economics 2014
Improving the quality of people’s lives is surely the ultimate aim of almost all government policy.
The report notes that data on subjective wellbeing gives us a more accurate picture of how policy translates into better lives for people – and pinpoints some of the shortcomings of the more traditional approach using GDP as a measure of social welfare.
But – it does not say what well-being is...nor how to measure...
In this talk focusing on the capacity of RDPE to deliver wellbeing.
RDPE part of the 2007-13 RDR – an EU programme.
We are interested in the potential effectiveness of rural development programmes for improving well-being in rural areas. Especially as focus of the RDPE is economic growth.
In particular our focus is on two aspects of the RDP for England: first to what extent can it deliver improvements in ‘wellbeing’ and secondly, how these improvements can be measured or evaluated.
We will address both these questions together based through a case study of CCRI work to evaluate two elements of the RDP for England over the period 2007-13
how do they work? and how do we measure their effectiveness?
_______________________________________________________
THESE ARE RELATED – and will be explored in rest of the talk.
EU rational for RDR
more than 90 % of the territory of the EU is "rural“ – with >56 % of the EU's population.
Landscape provides character, instils sense of place, value in terms of beauty, recreation, relaxation
Rural regions:
average income per head generally lower
skills base is narrower
service sector less developed.
caring for the rural environment incurs cost.
Provide raw materials
It acts as our lungs, and is therefore a battleground for the fight against climate change.
Our evaluation of Axes 1 & 3 of the RDPE 2007-13 was based on using Social return on investment as a tool to measure the value of programme impacts.
What we have done here is taken this a step further to explore whether SROI might be an effective tool for measuring wellbeing. This may seem odd as SROI measures change on a single dimension – monetary value – while well-being is clearly multi-dimensional…
However - SROI focuses on outcomes rather than specific project or programme outputs so it enables the user to define a wide range of effects for consideration.
For example, rather than simply measure the value of the number of jobs created by a project, it can explore a wider range of outcomes such as the quality of the employment, the effect on financial and personal wellbeing, and the effects of increased employment in a community in terms of social improvements.
In terms of assessing training and skills development, rather than just measure the number of people completing a training course it can measure changes in economic and social well-being among those who have benefitted from an improvement in knowledge or skills.
The SROI process:…
Application to Axes 1 & 3 of the RDP
England RDPE Focus on economic Growth – Axes 1 & 3 deliver this. Benefits exceed investment. In terms of both social and economic benefits.
SROI ratios – combined evaluative (to Dec 2012) and forecast outcome values over 5 yrs
Axis 1
Improving the competitiveness of agriculture and forestry
Large proportion of benefits from improvements in farm efficiency (reduced energy and nutrient input) and more efficient use of soil and water. Example is NW Livestock initiative which combined grants, training, and advice to farmers.
Large amount benefits relate to reduced carbon emissions and improved efficiency over 5 yr period.
Outcome values for livestock sector significant – large amounts invested in some regions through integrated programmes of advice, training, and grant support)
Forestry – relatively high outcome from improved access and carbon sequestration, relatively low outcome values from improvements in the wood supply chain
Axis 3
Improving the quality of life in rural areas and encouraging diversification of economic activity
Example is new tourism provision focused on small number of regionally strategic projects which created incentives for further private investment in supporting services.
Support for branding of local producers in a defined geographic area (Waveney Valley) to generate new micro-business
Local facilities such as community centres, recreational facilities and support networks.
Broader spread of benefits under Axis 3 across social as well as economic outcomes
Partly related to assumptions about:
numbers of beneficiaries in rural communities and numbers who volunteer (but we used conservative estimates)
Financial proxies – e.g. the proxy used to value volunteering
Tourism service provision also resulted in high outcome values – due to estimated numbers of tourists (and local residents) who benefit from improved service provision
Interest is in whether, or how well, SROI might measure dimensions of wellbeing.
Our analysis here is based on both the quantitative outcomes as measured in SROI and on qualitative assessment of improvements from interviews with beneficiaries, and those involved in implementation.
Green cells indicates a high contribution to wellbeing
Axis 1 this is education and environmental quality – which is where a lot of investment was focused – on improving soil and water management, and recuing energy and nutrient inputs, as well as improving skills in livestock management and animal welfare.
Education: xxxx number of people went through livestock training programmes, and the SROI analysis identified a range of associated outcomes including: the potential for improvement in income, etc. ……..
Axis 3 Strongest contributions were in education and social relationships. Again large emphasis in Axis 3 investment in training and skills improvement.
Social relationships is more interesting – some community focused projects had implicit goal of improving social relationships (e.g. Support for cooperative activities between villages)…
but in other cases measures we used indicated that this was the unintended outcome of a wide range of projects – for example - those where aim had been to support multiple activities along a supply chain, which had improved capacity for collaboration , or ‘catalyst’ projects where the investment had resulted in changes in thinking and behaviour in relation to a resource. One example is building of a new wild play area at a forest visitor centre which not only improved the financial situation of the centre but altered the way schools in a wide area used the forest and created new social relationships between disabled groups, local communities and the forest resource.
Yellow cells indicate a lower level of contribution to the dimension of well-being. So for example – Axis 1 undoubtedly contributed to improvements in material living among beneficiaries, and reduced insecurities through making business operations more financially sustainable. Though our data need more analysis to determine the level of contribution
White cells were areas where it was not clear on the level or extent of a contribution. For example - Axis 3 it was not clear to what extent investment had improved material living or environmental quality – though there was some evidence of change in some communities – partly due to the limits of the SROI approach and the way we identified outcomes and indicators.
For health – Axis 3 projects showed some clear evidence of improvements in psychological well-being and physical health as a result of improved access to community and recreational facilities. No clear evidence that axis 1 projects improved health.
Only 1 negative impact – in terms of political voice and governance. This is due more to change in delivery of the RDPE than anything else. In 2010 the programme was centralised with a loss of regional control – and this was keenly felt in particular by farmers and others in areas where there had been a large investment in development of integrated livestock improvement programmes (i.e. NW and SW). Respondents reported a loss of control over programme delivery and less understanding about the local situation on the part of decision makers.
Two final issues to address – how effective are RDPE programmes as delivering well being
And how might SROI be used to measure change in well-being.
1st question
– results suggest RDPs can be effective tools for improving well-being. The analysis revealed that several dimensions of well-being were improved by Axes 1 & 3 projects. So the area of interest is in how might current and future RDPs be utilised to enhance well-being even further. The evaluation project suggested some specific areas where programme benefits - and thus potentially well-being - could be enhanced:
Designing RDPs for improving delivery of well-being
Timing & continuity –to do with programming cycle
Timing – in some cases (e.g. SW region) development taking too long leaving little time for delivery;
Continuity of support – essential for some community measures – otherwise human and social capital that has been developed is lost
Catalyst projects – prime example is Camgrain in East Anglia – a very large project for improving grain storage had huge economic benefits for those involved as well as knock-on environmental benefits (reduction in fossil fuel consumption) and freeing up buildings and land across several counties for other uses. (£4 million – but benefitted 400 farmers)
BUT – projects don’t have to be large to have a ‘catalytic’ effect.
Floating jetty windermere – altered the way local authorities were thinking about local development plans and led to significant inward investment in improving the wider local economy; (30.5 mill)
wild play area in Whinlatter forest – reversed decline of visitors to a forest visitor centre, increased spending in the local economy, and changed the way schools think about and utilise the forest and the centre; £0.2 mill)
fishing lake for disabled in Lancashire – filled a gap – benefits across very wide geographic area (Less than £100,000)
Value of local knowledge in delivery and engagement – understanding local businesses and supply chains avoids displacement and deadweight – creates more opportunities for cooperation and adding value through for example – integrated programmes across a supply chain. Also enables tailoring of project to local context and identification of potential catalytic projects.
Investment approach – e.g. Support wide range of local businesses or support smaller number of regionally strategic projects – which then create incentives for further private investment. (e.g. SW tourism; E. Anglia water reservoirs)
e.g. Support a few large regionally import projects such as Camgrain, or larger number of locally important projects?
Finally – the potential of SROI as a tool for measuring well-being.
SROI a monetary based approach
The advantage of a uniform scale of measurement is in enabling comparison – across beneficiary types, projects, programmes and areas.
The advantage of the approach is twofold:
1st it measures outcomes and not outputs. So the user is free to identify a wide range of effects – both intended and unintended.
2nd it is not based solely on objective measures of value (e.g. investment in machinery increases output and thus business income). It does require qualitative understanding of the outcomes of investment impacts on individuals and communities. It requires the user to explore the range of effects along a number of outcome dimensions which might include behavioural change, household improvements, business implication and wider community benefits. These have to be identified and measured at the personal level through in-depth interviews, and the results have to be validated through triangulation with views of other stakeholders and those with a broader view of change across larger areas.
Having said that however – there are difficulties with applying the approach:
It can be difficult to identify beneficiaries, and how they benefit, and there are potential dangers of double counting of multiple benefits received by individuals
Measuring programme ‘reach’ – numbers of recipients – at programme level can be difficult – direct and indirect beneficiaries, all members of a community or only some, what constitutes a ‘community’?
What about projects that change mindsets (e.g. Floating jetty on Windermere that altered the way local authorities and developers thought about future economic development)?
Outcome identification – what are the wellbeing benefits?; short term impacts easier to identify than the long-term; direct and indirect benefits; double counting issues – what about an investment that increases financial sustainability of a business – can be economic improvements, improved confidence, more trust in others, more likelihood of further investment.
Capturing training benefits is difficult – outcomes difficult to ascertain and multiple ways benefits might be manifested (e.g. Through improvements in farming, resource use, competitiveness, confidence, potential to collaborate, etc.). Do simple scales capture the depth and complexity of change?
Selection of financial proxies – tend to be indicative of broad outcomes (e.g. Total annual variable input costs per farm as a proxy for cost savings; average hourly volunteer rate for measuring community development). More work is required to explore links between financial proxies and outcomes. Social benefits of community regeneration not easy to measure (used volunteer time as proxy)
In conclusion
SROI offers potential for measuring changes in well-being.
If more of the dimensions, which have been identified by the ONS as contributing to wellbeing were incorporated into the measurement of outcomes then it might become a more effective tool.